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1. Introduction 
1.1 Background to the Study 


The research that is the focus of this volume refers to an action research (AR) 
project regarding the English for Specific Academic Purposes (ESAP) course 
that I had been teaching at the Faculty of Economics and Management at the 
Free University of Bozen-Bolzano in South Tyrol, Italy, to undergraduate eco- 
nomics students on two undergraduate programmes, Economics and Manage- 
ment (E&M) and Politics, Philosophy and Economics (PPE), for several years 
prior to the study being undertaken. The research formed the basis for my 
doctoral studies, and some findings have already been published in abridged 
forms (Prior, 2018, 2020, 2021) as have some parts of the literature review 
(Prior, 2023). Although this volume contains some parts of these published 
articles with the editors’ permission, it is significantly more comprehensive 
since it focuses on the entire project and as such describes in greater detail the 
research process undertaken and the findings generated through the AR cy- 
cles. 

The ESAP course had been conceived in 2001 as an integrated credit-bear- 
ing mandatory language course for the students, whose degree programmes 
are characterised by the Faculty’s very specific teaching model that uses three 
languages of instruction, German, Italian and English, which tend to be dis- 
tributed equally over the subjects offered in each undergraduate degree pro- 
gramme. This study model makes the University one of the very few univer- 
sities in Europe that require undergraduate students to follow programmes 
using three languages of instruction.! 

Despite the University having a very specific language model, there is no 
formal university-wide policy as to how the three languages are distributed 
in degree programmes nor as to which subject a language is allotted. How- 
ever, the Faculty of Economics and Management distributes the languages as 
equally as possible over the subjects offered in their degree programmes and 


1 According to the Free University of Bozen-Bolzano’s website, studying with the 
trilingual model of the university is “a unique opportunity in Europe”. See 
https://www.unibz.it/en/services/language-centre/study-in-three-languages/ 
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so, as an example, Economics is taught and examined in English, private law 
in Italian and Financial Risk Management in German. Since the programmes’ 
subjects are only offered in one language and are not doubled up, this means 
that approximately two-thirds of the courses are taught in a student’s second 
(L2) or third language (L3). English is therefore used as a medium of instruc- 
tion, as is German and Italian, but the difference is that almost all students 
have either German or Italian as their first language (L1); in fact recent data 
show that of the total number of students enrolled in the E&M and PPE degree 
programmes at the Faculty, 37.06% have German as their L1, 59.1% have Ital- 
ian and 2.8% have another L1 other than German and Italian. However, only 
a further 1% have English as their L12. Therefore the vast majority of students 
following a course using English as a Medium of Instruction (EMI) have Eng- 
lish as their L2 or even their L3, with varying proficiency levels (Prior, 2021). 

EMI has been defined as “the use of the English language to teach aca- 
demic subjects in countries or jurisdictions where the first language (L1) of the 
majority of the population is not English” (Dearden, 2014, p.4). Although the 
English used in this context complies with this definition, as a medium of in- 
struction, however, it differs from Italian and German given that South Tyrol 
is a German-Italian bilingual area of Italy. As a result of this, the EMI courses 
at the University are held almost exclusively by lecturers who are non-native 
speakers of English, a feature which has been shown to be typical of Italian 
university contexts (Costa, 2013), but which again differs from the courses 
where German or Italian are used as a medium of instruction. These are almost 
without exception held respectively by lecturers with German or Italian as 
their L1. 

Given these specific considerations related to the Faculty’s teaching 
model therefore, the ESAP course, which runs concurrently with other sub- 
jects taught in English, had been conceived to provide extra skills practice and 
language input in English to assist in the study of those other courses using 
EMI (Prior, 2021). ESAP is a branch of English for Specific Purposes (ESP), 
which itself is a branch of English language teaching (ELT). ESP as a concept 


2 Data provided by the Student Secretariat at the Free University of Bozen-Bolzano on 
3 August 2022. 
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first appeared in the 1960s when global developments in the scientific, eco- 
nomic, and technological fields created a huge demand for specific English 
rather than the general English for no particular purpose that had traditionally 
been the focus of much language teaching and learning. Much teaching of 
English therefore started to shift towards a more functional syllabus which 
would provide language and skills training for specific professional and aca- 
demic needs in order “to help language learners cope with the features of lan- 
guage or to develop the competencies needed to function in a discipline, pro- 
fession, or workplace” (Basturkmen, 2005, p. 6). ESP is therefore distinct from 
general English and is defined as “an approach to language learning, which is 
based on learner need” (Hutchinson & Waters, 1987, p. 19). However, as far 
as the ESAP course of this study is concerned, no real needs analysis had ever 
been undertaken. This meant that the design of the syllabus had not taken into 
consideration the learners’ language needs and thus did not comply with this 
fundamental principle of ESP. 

An action research project was therefore initiated which used the Plan — 
Action — Observation — Reflection cycle (Kemmis & McTaggart, 1988) in order 
to conduct an initial wide-ranging needs analysis to investigate these needs 
from the perspective of the main actors in the EMI courses, the students and 
the lecturers whose language of instruction was English. These findings were 
then employed to create a much more detailed revised syllabus for the course, 
which was then successively modified over two further AR cycles covering 
three academic years in total. 

The desire to use two different sources for data collection was due to the 
pragmatist worldview that governed this study, which originates from my 
background as a teacher who has been designing and teaching ESP courses at 
university level for over twenty years. Further, my ontological perspective is 
that all relevant voices involved in a context have the right to be heard. Using 
the two different sources for the data would allow me to investigate the situ- 
ation more comprehensively because I would have gathered views from both 
sides of the classroom. These findings could then be triangulated in order to 
“achieve fuller understanding of [the] target phenomenon” (Dérnyei, 2007, 
p. 164), which would allow me to design a syllabus that could be as relevant 
as possible. Given the desire to include these two data sources, therefore, a 
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convergent parallel mixed methods research design was used in the quantita- 
tive and qualitative data collection and analysis stages (see Section 4.3) and 
then the results were integrated and discussed to verify the findings from both 
datasets as well as to explore different types of questions from each dataset. 

Before this research project had been initiated, the ESAP course had used 
a skills-based approach to designing its syllabus, and the course had been 
mostly teacher-fronted due to the large numbers of students attending the 
course (Sometimes reaching 80 in a class), although attendance was not com- 
pulsory. Since the students had to use various English language skills in their 
EMI courses, and on the findings from the initial needs analysis, the skills ap- 
proach to the syllabus was to be maintained. However, there was also the firm 
intention to provide each individual student with significantly more opportu- 
nities to engage in more relevant skills practice than had previously been the 
case in the teacher-fronted course. In order to combine these two aspects, ele- 
ments of a product approach to syllabus design, the skills-based approach, 
were blended with a process approach to syllabus design where negotiation 
would be used in the classroom to provide more opportunities for that skills 
practice and would allow the students to engage more actively in the decision- 
making aspects of their course. My years of experience have led me to believe 
that my learners should be allowed to contribute to the organisation of their 
course, which is often not the prevailing orthodoxy in the traditional educa- 
tional practices that characterise my work context. 


1.2 The Significance of the Study 
and Contribution to Knowledge 


Since Breen’s seminal article on syllabus design in 1987, the idea of blending 
product and process approaches has been posited, referred to variously as 
“multi-dimensional” (Breen, 2001), “hybrid” (Long, 2005c) or “eclectic” 
(Clarke, 1991) syllabuses. However, even though “no one approach can be re- 
sponsive to learners’ needs” (Graves, 2008, p. 161), there are no accounts of 
intentionally blending these approaches to syllabus design in the literature. 
There are a few accounts of process syllabuses that have been adopted in class- 
room settings around the world, but these accounts tend to focus on using a 
process approach instead of using a product approach. Moreover, many of 
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these admittedly few accounts often refer to a myriad of problems in the im- 
plementation of the process syllabus. Therefore, an account of an intentionally 
blended syllabus that has positive outcomes does not seem to appear in the 
literature. 

A further significant aspect of the study is the use of extra credit exercises 
(ECEs) that were used in AR Cycle 3 to incentivise attendance so that the class- 
room skills and language practice could be beneficial to as many students as 
possible. This use of ECEs in university language classrooms in Italy was the 
first account to be documented in the literature (Prior, 2018). 

This research contributes to existing knowledge on syllabus design and 
eclectic approaches to syllabus design for ESP in tertiary settings, in particular 
in the Italian context where research in this field is still in its infancy (see also 
Costa & Mastellotto, 2022). It also makes a more recent contribution to the 
work of other researchers who have undertaken classroom-based research on 
practical implementations of negotiated syllabuses, such as Budd & Wright 
(1990) and Serrano-Sampedro (2000), and particularly in university settings 
such as some of the accounts in Breen & Littlejohn (2000a), including Martyn 
(2000), Newstetter (2000) and Sokolik (2000), as well as in Boon (2011) and 
Wette (2011), among others. 

Further, this research contributes to the knowledge on how using a nego- 
tiated syllabus can be associated with providing learners with greater auton- 
omy in their learning, as was reported by Bloor & Bloor (1988) and Cotterall 
(2000). It has also been suggested that using negotiation in the classroom pro- 
vides more opportunities for skills practice (Serrano-Sampedro, 2000) and so 
this research aims to demonstrate that this can indeed be the case and that this 
practice is beneficial for improvement in the skills. 

Finally, the research documented in this volume contributes to existing 
knowledge on using action research with multiple AR cycles in English-lan- 
guage classrooms by providing a detailed account and evaluation of this 
teacher research. Although there are accounts of AR studies by teacher prac- 
titioners working in classrooms, they tend to be limited in number (Burns, 
2005b) and many date back to more than ten years ago (Burns & McPherson, 
2017) so this study is a more recent addition to the literature. Moreover, it 
provides insight into the iterative nature of AR and how the multiple cycles 
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that characterised this study enabled “initial insights and findings to give way 
to deeper, new but related questions” (Burns, 2005b, p. 67), which led to 
greater rigour in the research. Using multiple cycles also allowed me to inte- 
grate the contributions received from the learners, who are normally excluded 
a voice in the decision-making process of their learning, so that the content 
and organisation of their courses in the successive cycles could be improved. 


1.3 Research Questions 


The research questions were developed to reflect both the AR tradition and 
the convergent parallel mixed methods research design. The first two research 
questions are presented as mixed methods questions whereas the subsequent 
three questions were designed to be investigated through taking practical 
measures in the AR cycles. 


The first two questions concern the needs analysis that was required to be un- 

dertaken in order to redesign the ESAP syllabus: 

1. What are the English-language skills needed by economics students at 
this trilingual university as perceived by the main “actors”, i.e. students 
and lecturers? 


2. What skills practice should be maintained or enhanced in the syllabus? 


These are mixed methods questions, as they required access to two different 
data sources, the students and the EMI lecturers, and the answers had to be 
sought through the merging of the analysed data from the two datasets, which 
was done through a “side-by-side comparison” (Creswell, 2014, p. 222). 


The other research questions regard the action that needed to be taken in the 

classroom once the needs analysis had been conducted, reflecting on the ac- 

tion undertaken and evaluating it: 

3. How can a predominantly product syllabus that is skills based benefit 
from the integration of a process approach to syllabus design? 

4. What elements of the syllabus can be negotiated with the learners consid- 
ering the constraints of this particular context and experiences in other 


contexts? 
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5. What evaluation methods can be devised to ascertain the effectiveness of 
this proposed syllabus for each individual learner? 


As the focus of AR is to “bring about changes and, even better, improvements 
in practice” (Burns, 2010, p. 2), these questions could only be investigated 
through at least two AR cycles, since they required reflection on the action 
undertaken both during and at the end of the course and adjustments made 
when necessary. Moreover, given the learner-centred focus of the syllabus and 
the desire to make it as flexible and as practical as possible so that it could be 
used with different groups of students, the use of multiple AR cycles would 
allow for a more rigorous approach to this evaluation through multiple data 
collection stages and methods. 


2. Literature Review 


2.1 An Overview of Syllabus Design in ELT 


As summarised in Prior (2023), any literature review of syllabus design in ELT 
should begin with Breen’s two-part state-of-the-art article (1987a; 1987b). His 
work can be considered a significant contribution to the paradigm shift in syl- 
labus design that was occurring at the time as he examines not only the formal 
and functional syllabus types, which were undeniably the most popular types 
of syllabus at the time, but also the task-based and process syllabus. These had 
begun to emerge during the eighties and were influenced to a great degree by 
Krashen’s (1985) observations that language acquisition came from the 
learner’s ability to understand meaning from language input rather than from 
studying the grammar of a language. Breen argues that various developments 
in language research prior to his article being published also contributed to 
the paradigm shift. These developments concerned changes in the way as- 
sumptions were held about language, teaching methodology, learner contri- 
bution and how planning for teaching and learning was conducted (Breen, 
1987b, p. 157). 
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It is evident that Breen was accurate in identifying a shift in syllabus design 
in the late eighties due to this belief being shared by others (e.g Graves, 2008), 
the number of other influential works about innovations in syllabus design 
around that time (Candlin, 1984; Yalden, 1987; Prabhu, 1987; Nunan, 1988a) 
and the development of ESP approaches to teaching which stressed the neces- 
sity to identify learners’ needs in order to plan a language syllabus 
(Hutchinson & Waters, 1987; Master, 2005). It is beyond the scope of this lit- 
erature review to summarise all the developments Breen refers to in his article, 
but the changes in syllabus design referring to product and process ap- 
proaches will be referred to due to their significance for the research project 


in this volume. 


2.2 Product Approaches to Syllabus Design 


Up until the end of the seventies when communicative language teaching 
(CLT) started gaining acceptance, a syllabus had traditionally focused on the 
linguistic elements that were deemed necessary to be taught and learnt, which 
were the grammar and the skills of listening, reading, writing and speaking 
(Breen, 2001, p. 152). This type of syllabus was known as the formal syllabus, 
but it is also commonly referred to as the grammatical or structural syllabus, 
and it is a syllabus based on the assumption that language learning should be 
strictly sequenced based on moving from grammatical or linguistic simplicity 
to greater complexity. The main aim tends to focus on producing correct 
grammatical structures and that “the skills be worked upon in a sequence 
from the receptive to the productive” (Breen, 1987a, p. 85). However, Nunan 
asserts that “grammatical complexity does not necessarily equate with learn- 
ing difficulty” (1988a, p. 28), and the sequencing of language into grammatical 
chunks implies that there is a finite amount to learn. Additional criticism di- 
rected at grammatical syllabus concerns the fact that language and skills learn- 
ing does not naturally occur in such a structured, sequenced way and basing 
a syllabus purely on grammar and/or skills entirely misses all the other as- 
pects that make language as complex as it is. However, this sequencing of lan- 
guage makes a grammatical syllabus simple to package and so it “has a strong 
theory of language, but not a strong theory of learning” (Graves, 2008, p. 160). 
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Perhaps the greatest criticism of the grammatical syllabus, however, was that 
it did not take into consideration the communicative purposes of language. 
The greatest reaction to the grammatical syllabus was the functional/notional 
syllabus (Wilkins, 1976) that developed in the seventies along with the publi- 
cation of the Council of Europe’s Threshold Level (van Ek, 1975), which was 
an attempt to standardise the learning and teaching of some of the main Eu- 
ropean languages. This type of syllabus bases its content on the functional 
skills and notions that the learners need to master, rather than just the gram- 
matical elements of the language. Hutchinson & Waters state that “functions 
are concerned with social behaviour and represent the intentions of the 
speaker or writer, for example, advising, warning, threatening” and notions 
are the “categories into which the mind and thereby language divides reality, 
for example, time, frequency, duration” (1987, p. 31). However, if a syllabus is 
based on the functions and notions that learners need, this implies that a phase 
must be integrated into syllabus design where these needs are assessed. This 
shift towards a functional approach therefore made it necessary to incorporate 
needs analysis into syllabus design, which was clearly unnecessary in gram- 
matical syllabus design as the starting point was the underlying structure of the 
language itself (Graves, 2008, p. 160). The functional approach to syllabus 
design, with this emphasis on needs analysis, was the catalyst that allowed ESP 
courses to evolve dramatically in the 1980s. It was assumed that ESP learners 
had already followed a grammatical syllabus, and so had a good understanding 
of the deep-rooted structure, but now they needed “to learn how to use the 
knowledge they already have” (Hutchinson & Waters, 1987, p. 32). Master (2005, 
p. 101), in his survey of ESP topics reported in published papers, notes that the 
opinions of the learners “comprised an essential element of the needs analysis”, 
but once the importance of needs analysis had been clearly established, further 
research into ESP syllabus design fell significantly in the 1990s. 

The formal and functional types of syllabus detailed above were seen as 
similar approaches to syllabus design in that learners were expected to gather 
often de-contextualised and separate language items, whether they be forms 
or functions, before being able to apply the language in any real communica- 
tive context. These two types of syllabus were described by Breen as proposi- 
tional plans, in which “language knowledge and capabilities regarded as the 
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appropriate outcomes will be organised and presented in the plan as things 
which are systematic; perhaps based on logical formulae, structures, net- 
works, rules, or schemas” (1987a, p. 85). Nunan (1988a), however, when refer- 
ring to these formal and functional syllabuses uses the term product-oriented 
syllabuses, which he describes “are those in which the focus is on the 
knowledge and skills which learners should gain as a result of instruction” 
(Nunan, 1988a, p. 27), in other words, “learning is supposed to result in a 
product — a set of knowledge and skills” (Graves, 2008, p. 160). The two defi- 
nitions relating to propositional plans and product-oriented syllabuses are 
very similar to how synthetic syllabuses were described (Wilkins, 1976), where 
the language to be taught is broken down into its composite parts: lexis, syn- 
tax, morphology, pronunciation and is taught in ever-increasingly sophisti- 
cated stages until the language has been learned. Learners then have to “re- 
synthesize the language that has been broken down into a large number of 
small pieces” (Wilkins, 1976, p. 2) when they need or want to use the language 
for communicative purposes. Nunan, elaborating on his product-oriented syl- 
labuses, makes the point that although synthetic syllabuses were initially 
equated just with the grammatical syllabus, all product-oriented approaches 
to syllabus design are generally regarded as synthetic (1988a, p. 28). For the 
purposes of this volume, any further reference to this type of syllabus will 
make use of the terminology product approach in order to avoid confusion. 

Despite the advent of CLT at the end of the seventies, which certainly 
heralded a backlash against grammatical syllabuses, product approaches are 
still prevalent in English-language teaching and many commercial course- 
books are still based on traditional formal syllabuses (Harmer, 2003; Wette, 
2011). This is because “educational practice is concerned with the achievement 
of certain desired end states” (Eisner, 1992, p. 302) and so a product approach 
to syllabus design can be more easily evaluated and thus more attractively 
presented and sold. This view of syllabus types therefore regards what actu- 
ally happens inside the classroom solely as a means to achieving the end prod- 
uct and what happens in the classroom, or the processes themselves, tend to 
be overlooked (Prior, 2023). 
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2.3 Process Approaches to Syllabus Design 


In the second part of his article, Breen examines how the paradigms of sylla- 
bus design had been changing during the eighties. As mentioned at the begin- 
ning of this review, he identifies four frames of reference which affected how 
syllabus design had started to change at the time: language; teaching method- 
ology; learner contributions; how we may plan for teaching and learning, and 
particularly how these aspects had started to merge (1987b, p. 160). These 
changing assumptions about language teaching and learning largely contrib- 
uted to the development of alternative syllabuses, which he identifies as the 
task-based and process syllabuses. 

Before discussing these two types of syllabus in more detail, it is important 
to understand how these syllabuses compare to formal and functional sylla- 
bus, which we have seen are product approaches to syllabus. Breen describes 
task-based and process syllabuses as process plans as opposed to propositional 
plans and states that they 


represent how something is done. They will seek to represent knowledge of how 
correctness, appropriacy, and meaningfulness can be simultaneously achieved during 


communication within events and situations (Breen 1987b, p. 160 — original italics.) 


This idea of focussing on the process, rather than the product in the formal or 
functional syllabus, was also highlighted by Wilkins (1976), who used the 
term “analytic” syllabus to describe a more holistic syllabus where learners 
are given samples of the target language in an ungraded state and they ana- 
lyse it to gain meaning. Wilkins states that analytic syllabuses “are organized 
in terms of the purposes for which people are learning language and the kinds 
of language performance that are necessary to meet these purposes” (1976, p. 
13), which was echoed more than twenty years later by Markee who states: 
“learning is organized in terms of the social purposes that learners have for 
learning the target language” (1997, p. 16). Wilkins had initially envisaged that 
the functional syllabus would be an analytic syllabus but as the focus of the 
functional syllabus is the product, i.e. the skills and functions needed by the 
learner, it has been generally accepted that it is a synthetic syllabus (cf. Nunan 
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1988a). As a further exemplification of the confusion that can arise when dis- 
cussing the terminology relating to approaches to syllabus design, Nunan 
(1988a) uses a different term when referring to task-based and process sylla- 
buses and calls them “process-oriented syllabuses”. Despite using this term, 
he agrees with the broadly accepted view that these types of syllabuses focus 
on the “processes through which knowledge and skills might be gained” 
(Nunan, 1988a, p. 41). 

An advantage of taking a process view of syllabus design is that it takes 
content (the syllabus) and merges it with methodology (how it should be 
learnt). In synthetic syllabuses, which are product-oriented, there tends to be 
a separation of the syllabus and methodology (Nunan, 1988a; Breen, 2001; Lit- 
tlewood, 2009). However, in analytic syllabuses the fact that the what (content) 
and the how (methodology) are inevitably interconnected is explicitly recog- 
nised. Further, these types of syllabuses do not regard classroom activities as 
a mere means to achieve a linguistic goal, but rather the language is the means 
to undertake activities in the classroom. This assumption is represented in 
what Breen refers to as process plans, the task-based syllabus and the process 
syllabus being the two types he discusses (1987b), otherwise known as the 
process-oriented syllabus (Nunan, 1988a; Graves, 2008). 


2.3.1 The task-based syllabus 


The task-based syllabus can be regarded as the most well-known (Graves, 
2008) of the two types of process plans that Breen (1987b) identifies and has a 
dual-focus: to enable learners to develop communicative competence and 
share meanings but also to achieve something tangible while the language is 
being used (Breen, 1987b, p. 162). The tasks used in the classroom aim to “en- 
gage the same abilities which underlie communication itself” (Breen, 1987b, p. 
162 — original italics) and because the learning process itself is seen as the con- 
tent, this type of syllabus neatly blends content and methodology. Long & 
Crookes state that the selection of tasks to be used within this syllabus must 
be undertaken once a needs analysis has been conducted so that the “real- 
world target tasks learners are preparing to undertake” can be replicated (1992, 
p. 44 — original italics). Over the years, however, the definition of task has 
changed somewhat (Nunan, 1989, p. 5). The definition has ranged from 
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Breen’s suggestion that tasks used in the learning process should exemplify 
target language communication in the target situation (1987b, p. 162), to Wil- 
lis’s much broader approach where “tasks are always activities where the tar- 
get language is used by the learner for a communicative purpose (goal) in or- 
der to achieve an outcome” (1996, p. 23). Given the popularity of certain task- 
based coursebooks that have come onto the market over the past decade or so, 
it seems that this broader definition of what a task is has taken hold. 

Despite the popularity of the task-based syllabus over the years, however, 
it has remained controversial, mainly because there has been little research 
that substantiates many of the claims of its advocates (see for example 
Seedhouse, 1999, p. 149; Willis & Willis, 2001, p. 176; Harmer, 2003, p. 173). 
Further, as seen above, the definition of task is “fraught with problems” (Lit- 
tlewood, 2004, p. 325) and can range from a simple classroom activity to some- 
thing “so broad as to include almost anything that involves learners doing 
something” (J. Richards, 2001, p. 162). The other main criticism of task-based 
learning is that it can favour fluency over accuracy (J. Richards, 2001, p. 162) 
and even if a task has been designed with a focus on meaning, for example, it 
is usually down to the individual learner as to how to interpret the task, and 
so s/he may focus entirely or mainly on form (Littlewood, 2004, p. 324). More- 
over, teachers have expressed concern about using task-based learning in their 
classrooms. In their more recent study, Zheng & Borg (2014, p. 208) report 
some of the challenges task-based learning poses to teachers, including it be- 
ing more time-consuming, it can be problematic to implement in large classes 


and it often does not reflect how English is assessed in examinations. 


2.3.2 The negotiated (process) syllabus 


The other “process plan” that Breen identifies is the process syllabus, which, 
like the task-based syllabus, is an analytic syllabus that focuses on the how 
rather than on the what. However, a process syllabus is more than a task- 
based syllabus as it provides “a bridge between content and methodology and 
[offers] a means whereby the actual syllabus of a classroom group may be 
made more accessible to each of its members” (Breen, 1987b, p. 166). He sees 
the process syllabus as a means to transfer the creation of a syllabus from the 
designer to the teacher and crucially also to the learners so that they “create 
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their own syllabus in the classroom in an on-going and adaptive way” (Breen, 
1987b, p. 166). An underlying assumption is that because teachers who have 
to work with a syllabus necessarily have to interpret it and adapt it to each 
specific group of learners, which makes a prepared syllabus redundant as 
soon as it has been produced, a type of syllabus should be designed which 
explicitly takes this into account and can (or should) be reinterpreted (Breen, 
1987b, p. 166). Other research has confirmed that teachers will often deviate 
from a prescribed syllabus or negotiate aspects of the learning process with 
learners to cater better to learners’ needs despite having to adhere to imposed 
curriculums (Linder, 2000; Serrano-Sampedro, 2000; Wette, 2011). Therefore 
the process syllabus 


provides the framework within which either a predesigned content syllabus would 
be publicly analysed and evaluated by the classroom group, or an emerging content 
syllabus would be designed (and similarly evaluated) in an ongoing way. (Breen, 
1984, p. 55) 


Breen & Littlejohn state more simply that a process syllabus provides a frame- 
work for decision-making in the classroom (2000c, p. 29), which is undertaken 
by both the teacher and the learners through negotiation. Negotiation, accord- 
ing to Breen & Littlejohn, is “discussion between all members of the classroom 
to decide how learning and teaching are to be organised” (2000b, p. 1). This 
emphasis on making decisions in a collaborative way between the teacher and 
the learners in order to organise the course was referred to as an essential 
component of learner-centred approaches to syllabus design as presented by 
Nunan: 


While a learner-centred curriculum will contain similar elements and processes to 
traditional curricula, a key difference will be that information by and from learners 
will be built into every phase of the curriculum process. Curriculum development 
becomes a collaborative effort between teachers and learners, since learners will be 
involved in decisions on content selection, methodology and evaluation. (Nunan, 


1989, p. 19) 
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The term learner-centredness has been used to refer to various views on lan- 
guage teaching, ranging from humanistic perspectives that regard learners as 
complex beings rather than “just” language learners, to teaching methods de- 
signed to foster learner autonomy (Tudor, 1993). However, Nunan’s view that 
learner-centredness can be understood as the involvement of learners in the 
design of their own learning, often through negotiation, has often been the 
accepted definition adopted in studies of learner-centred approaches to teach- 
ing (Davies, 2006; Zhang & Head, 2010). 

One of the first times the term negotiated syllabus was used was by Clarke 
(1991) who investigated using negotiation with learners to design their learn- 
ing programmes. He used it as a synonym for Breen’s process syllabus, and 
so it would be useful to clarify terminology at this stage. Breen (1987b) uses 
process plan as a superordinate category that covers both the task-based and 
process syllabus. However, Breen & Littlejohn (2000a; 2000b) in their later vol- 
ume tend to use the term negotiated syllabus as a synonym for process sylla- 
bus and use both terms interchangeably to refer to the same type of syllabus. 
Therefore, from hereon in, to avoid confusion and whenever necessary, when 
referring to process approaches to syllabus design, in the sense of process 
plans as Breen (1987a; 1987b) names them as discussed above, the term pro- 
cess approach will be used. When referring to the negotiated (or process) syl- 
labus, the terms process syllabus and negotiated syllabus will continue to be 
used interchangeably to refer to the same type of syllabus that “allows full 
learner participation in selection of content, mode of working, route of work- 
ing, assessment and so on” (Clarke, 1991, p. 13). 

As summarised by Prior (2020), negotiation that takes place in the class- 
room can relate to the content, the language learning procedures, the goals 
and even how these or other aspects are assessed. Breen & Littlejohn see this 
process as a cycle where decisions are negotiated initially about one or more 
aspects relating to the classroom work in the initial stage, actions are taken to 
implement those decisions in the next stage and then there is an evaluation 
phase where both the learning outcomes and the process that led to those out- 
comes are evaluated. This negotiation cycle is reproduced in Figure 2.1: 
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Step 1 Negotiated decisions 
f Purposes Contents Ways of working Evaluation 
Why? What? How? How well? 
The aims of The focus of What resources will What should be 
classroom work classroom work, be used? Whenand the outcomes? 
for example how long will How will they be 
language areas, something be done? assessed? What 
topics, skills, Who will work with will happen with 
learning strategies who? How much the assessment? 
guidance will be 
available? 
Step 2 | 
Action(s) 


Undertaken on the basis of the negotiated decisions at Step 1; 
for example: tasks chosen and completed, plans made, 
evaluation procedures worked out 


Step 3 Evaluation 


e of learning outcomes: achievements and difficulties 


e ofthe process itself in relation to outcomes: appropriateness of 
purposes, contents, ways of working, evaluation and action taken 
at Step 2 


Fig. 2.1 — The negotiation cycle. From Classroom Decision-Making: Negotiation and Process Syl- 
labuses in Practice (p. 32), by M. P. Breen and A. Littlejohn. Copyright 2000 by Cambridge Univer- 
sity Press. 


An aspect of negotiated syllabuses that is of prime importance is step 3 in the 
cycle; the role of evaluation and how this can inform and improve the learning 
process. Breen & Littlejohn believe that evaluating the actions proposed 
through negotiation is the most important feature of these types of syllabus 
(2000c, p. 33). Evaluation occurs through the group reflecting on the outcomes 
and the appropriateness of the decisions that were taken and the actions that 
occurred from these decisions. What is important about this evaluation phase 
is that “it generates essential information for teacher and students for the next 
cycle of decision-making” (Breen & Littlejohn, 2000c, p. 33). This closely re- 
flects Breen’s previous argument that process syllabuses are retrospective in 
that during the evaluation phase(s) the group can reflect on the decisions that 
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led to achievements and difficulties and in this way adapt the decisions for 
further phases in the learning process or for the next course (Breen, 1987b, 
p. 167). 

Consequently, a syllabus based on negotiation between the teacher and 
the learners involves those learners in the decision-making processes of a syl- 
labus and as such “gives voice to students in the management of their learn- 
ing” (Breen & Littlejohn, 2000b, p. 1). Slembrouck claims that “educational 
practices are responsible for gradually silencing youngsters” (2000, p. 148), so 
using a negotiated syllabus that allows learners to actively contribute to the 
decision-making processes that are involved in the design of their syllabus 
provides a more democratic approach to teaching and learning. Graves (2008) 
also refers to involving teachers and learners in the design of a syllabus in- 
stead of allowing specialists, who tend to be external to the classroom context, 
to undertake this task as they often work sequentially; first the curriculum is 
planned, then needs are analysed, then materials are written and then imple- 
mentation takes place. She objects to this system and states that “by putting 
the classroom at the end of the chain of decisions, it positions teachers — and 
learners — as recipients and implementers of received wisdom, rather than de- 
cision-makers in their own right” (Graves 2008, p. 151). As such, she sees the 
teacher as “a catalyst for curriculum change” (Graves 2008, p. 171), but also 
acknowledges the important role of the learners in the decision-making pro- 
cess so that what they themselves want to do and achieve is clearly reflected 
in the syllabus design. 

A process syllabus that promotes negotiation also enables each individual 
to bring to the learning process their own distinctive previous knowledge and 
experience and, with the support and feedback from the rest of the group, 
including the teacher, allows the learner to become “an active agent” in the 
learning process (Breen & Littlejohn 2000c, p. 24). If learners are active agents, 
they can “exercise autonomy on an equal footing with others in the group”, 
which “promotes a learner’s power of learning and interdependency in learn- 
ing when appropriate” (Breen & Littlejohn 2000c, p. 22). Consequently, given 
the fact that all learners are individuals with specific needs, with their own 
particular knowledge and proficiency in the language, a process syllabus can 
allow each learner to focus on what s/he feels is beneficial and appropriate for 
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his/her needs and interests during the learning process, yet with the support 
and input from the entire group and the teacher (Prior, 2023) 

Clarke refers to the pedagogical value of designing a negotiated syllabus, 
since the design of the syllabus is “no longer external to, or prior to, the im- 
plementation of the syllabus” (1991, p. 14). As learners are involved in the 
decision-making about the course, the design of the syllabus becomes the 
“most essential pedagogical component” and “a process concept, a dynamic 
and flexible dimension of the learning experience” (Clarke, 1991, p. 14). The 
pedagogical value of using negotiation in the classroom means that learners 
are provided with opportunities to engage in authentic communication. De- 
spite the pervasive presence of the internet nowadays, often many language 
learners only have contact with the target language in a classroom environ- 
ment, and in particular, they may only have opportunities to produce the target 
language in the classroom. Therefore, if they are given opportunities to nego- 
tiate over the syllabus and the learning process, this will allow them to engage 
in authentic communication with both the teacher and the other learners. By 
doing this, they are provided with opportunities to acquire some of the fun- 
damental skills and language that will be needed in other situations outside 
the classroom environment where negotiation is required. It has been argued, 
therefore, that through this negotiation, language learning can occur because 
negotiation is itself a communicative activity (Breen & Littlejohn, 2000a; Ser- 
rano-Sampedro, 2000; Waters, 2012). 

Although the idea of process syllabuses is appealing due to the learner-cen- 
teredness of the approach, this type of syllabus has not been adopted by main- 
stream ELT coursebooks, which, as mentioned previously, tend to promote a 
product approach, in which language is sequenced within a syllabus and tends 
to be dealt with separately, and which is more easily packaged than a process 
approach. Wette (2011, p. 137) observes that process approaches tend to be found 
in what she terms “low-constraint” contexts, where teachers have greater 
autonomy in decisions related to syllabus design and content. To a certain extent 
this observation contradicts Breen’s idea that the process syllabus offers a way to 
work around and within an existing syllabus (1987b, p. 166), an idea which is 
reiterated by Breen & Littlejohn (2000c, p. 34) who state that this type of syllabus 
is often used in conjunction with another, pre-designed syllabus or ready-made 
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materials. Although it can be safely assumed that many, perhaps even most, 
teachers are not in low constraint contexts and as such do not have a great degree 
of freedom to decide on classroom content, Breen’s contention that teachers and 
their learners interpret pre-existing or pre-designed syllabuses in any case makes 
the adoption of a process syllabus seem reasonable and even relatively workable. 
Wette (2011, p. 137) does make a valuable point, however, in that “pure” versions 
of a process syllabus are virtually non-existent, which is echoed by Breen & 
Littlejohn (2000c, p. 30) who state that “it would be highly unusual and inefficient 
for a classroom group to seek negotiated agreement on all of the major questions 
in every lesson, even if this was feasible”. Clarke (1991, p. 16) goes further and 
claims that it is “the radical nature of the Negotiated syllabus ... which is likely 
to confine the application of its full-blown or pure version to rather rarely 
obtained circumstances”. Markee (1997, p. 22) also refers to “the strong form” of 
a process syllabus, as he defines it, and suggests this type of process syllabus has 
probably never been implemented and that most practical applications are of a 
weaker form. In fact, there are very few documented examples of pure or almost 
pure forms of a process syllabus. One example that was documented was 
implemented in a Belgian university for an undergraduate English-language 
course, but this experience was characterised by a series of problems including 
erratic attendance, rejections of the tutor’s suggestions and fundamental dis- 
agreement amongst the students as to the contents and ways of working 
(Slembrouck, 2000). The problems were so multifaceted that the course was 
eventually discussed at the faculty board level due to student complaints and 
was abandoned after one semester. Another almost pure form is documented in 
the same volume but for a course for ELT professionals, and this experience also 
led to a series of tensions and difficulties due to the participants not being 
particularly proficient at negotiating and their expressing dissatisfaction at their 
apparent lack of progress through the course (Ivanič, 2000). 

Therefore, the practical applications of process syllabuses that have been 
documented tend to have been weaker forms (Budd & Wright, 1990; Breen & 
Littlejohn, 2000a) due to the impracticable nature of “pure” forms. However, 
Breen & Littlejohn (2000c) suggest that a “pure” process syllabus should not 
necessarily be the goal and that a process syllabus should rather be interpreted 


as a framework for decision-making. This would imply that the number and 


19 


Integrating Classroom-Based Negotiation Into a Syllabus 


type of decisions open to negotiation can and will differ greatly from context 
to context (Breen & Littlejohn, 2000c, p. 29). To illustrate this framework ap- 
proach, and to show how a process syllabus can be applied practically, they 
propose a “curriculum pyramid”, which depicts how decisions can range 
from the widest context possible, the entire curriculum, to the narrowest at 
the single task level (Breen & Littlejohn, 2000c, p. 35). 


A task 
A sequence of tasks 
A series of lessons / sessions 


A course 


A specific subject / language curriculum 


A wider educational curriculum 


Fig. 2.2 — The curriculum pyramid: levels of focus for the negotiation cycle. From Classroom Deci- 
sion-Making: Negotiation and Process Syllabuses in Practice (p. 35), by M. P. Breen and A. 
Littlejohn. Copyright 2000 by Cambridge University Press. 


If a process syllabus is regarded as a framework, such as the one illustrated 
here, this type of syllabus therefore becomes much simpler to enact in a prac- 
tical context and, of course, can be easily adapted from one context to the next. 

However, there are certain weaknesses of process syllabus that have been 
noted over time. Breen (2001, p. 156) himself questions the feasibility of using 
process syllabuses in certain contexts and highlights various factors that may 
negatively affect the adoption of a process syllabus, including student expec- 
tations and the conventions and expectations of the teaching context and the 
institution itself. The main perceived drawback of a process syllabus and the use 
of negotiation in the classroom is that it subverts the traditional institutional 
teacher-student role, which requires the teacher to make all the decisions 
concerning the teaching and learning process. Bloor & Bloor (1988, p. 65) refer to 
this attitude of learners who often “see themselves as operating in a passive role, 
just ‘being taught”, and see the teacher as the active member of the partnership. 
Budd & Wright (1990, p. 225) highlight this difficulty during their experience of 
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implementing a process syllabus and state “the apparent reluctance of the 
learners to become involved is almost certainly an outcome of the conflict that a 
shift in role relationships brings about”. Slembrouck (2000), in his account of a 
process syllabus not going to plan, also identifies this shift in roles as a reason for 
the failure of his course, as does Newstetter in the same volume, who found 
many of her students had an “inability to comprehend this system” (2000, p. 182). 
Learners’ ambivalence about their role and that of the teacher in the negotiated 
classroom was also reported by Boon (2011). Therefore, although a process 
syllabus strives to attain learners’ greater involvement in the learning process 
through the negotiation of decision-making, it is actually this very aspect that 
alienates many learners due to the overriding influence of the traditional model 
of learning where the teacher is expected to have all the answers and make all 
the decisions. Similar to these accounts in the literature, the problem of learner 
resistance to the redesigned ESAP syllabus was experienced in the study that is 
the focus of this volume. When the negotiated syllabus was introduced to the 
students in AR Cycle 2, only two students selected to adopt the redesigned 
syllabus, and this low take-up rate led to its unsuccessful implementation and its 
subsequent modification for the third and final action research cycle (details can 
be found in Chapters 6 and 7). 

The traditional model of learning can act as a barrier to the successful 
implementation of a negotiated syllabus, but Clarke (1991, p. 20) identifies the 
resistance emanating not from the learners but from the teachers themselves, 
since “the very idea of negotiation is likely to lead to that of anarchy and, ina 
sense, a dereliction of duty”. White (1988, p. 102) also has similar concerns and 
comments on what he believes would be the practical problems that some 
teachers would face if they had to use a process syllabus as this, in his view, 
would imply abandoning the coursebook, “the mainstay in many courses in 
which the textbook equals the curriculum”. It would be difficult, therefore, to 
imagine how a process syllabus could be implemented since the support of a 
coursebook “is a tradition that dies hard ... in many situations the coursebook 
is all the hard-pressed or underskilled teacher has to rely on” (White, 1988, 
p. 102). These concerns about teachers’ resistance to models of a negotiated 
syllabus are certainly valid as “teachers’ beliefs influence their instructional 


choices” (Borg & Al-Busaidi, 2012, p. 283), but they relate to strong or pure 
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forms of negotiated syllabuses that, as discussed, would most likely be un- 
workable in almost all educational contexts. Indeed, in their study on teachers’ 
attitudes to providing learners with more autonomy, one of Borg & Al- 
Busaidi’s key findings was that although teachers were generally positive 
about the desirability of learner involvement in decision-making processes 
about their courses, they were sometimes significantly less positive about how 
feasible this would be (2012, p. 286). Stevick also refers to the problems inher- 
ent with providing too much freedom for learners in the classroom and high- 
lights the role of the teacher as someone who “controls” what is happening in 


the classroom: 


If we, in our zeal to be “humanistic”, become too “learner-centered” with regard to 
“control”, we undermine the learner’s most basic need, which is for security. We 
may find that we have imposed our own half-baked anarchy on the class. Absence 
of structure, or of focus on the teacher, may well be all right in certain kinds of 


psychological training, but not in our classrooms”. (Stevick, 1980, p. 33) 


Barriers to the negotiated syllabus can also originate from the institutions 
themselves. Slembrouck states clearly that “classroom negotiation entails that 
one seeks to restore the speech of the students by reintroducing their right to 
plan and assess the educational process” (2000, p. 148), but it has also been 
claimed that “the student as an active learner is not very welcome in most 
sectors of universities” (Levin & Greenwood, 2001, p. 104). Consequently, the 
fact of giving the learners a voice in the decision-making processes of their 
courses can disturb the institutional status-quo. This aspect was not a feature 
of my study as, being in a “low-constraint” context (Wette, 2011, p. 137), Ihave 
ample freedom to design the course, especially regarding methodology and 
course contents. 

Because a negotiated syllabus can be regarded as “deviant” (Clarke, 1991, 
p. 20) from the established models of traditional educational contexts, its main 
weakness seems to be the resistance shown towards it by the two main actors 
involved in its design and implementation, the teacher and the learners, as 
well as from the institution itself. Consequently, in view of this resistance, an 


alternative approach would need to be considered where the benefits of using 
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negotiation in the classroom could be exploited, but the more radical and un- 
workable aspects that have created such an opposition to process syllabuses 
would be avoided. This alternative approach would imply using a blend of 
elements that characterise a process syllabus with other aspects from a more 
product-based syllabus to mitigate the perceived radical nature of the process 
syllabus. 


2.4 The Blended Syllabus as the Next Stage 
in Syllabus Design 


A syllabus represents the “assumptions about the nature of language and lan- 
guage learning” (Nunan, 1988a, p. 29), and because of the changing attitudes 
and approaches to language learning, the area of syllabus design in ELT has 
experienced an abundance of developments and disagreements over the 
years. Particularly, syllabus design has tended to base itself on dichotomies, 
to which this literature review has alluded. Breen (1987a; 1987b) had already 
acknowledged a clear difference in paradigms used in syllabus design in his 
two articles on syllabus design, which this literature review used as its starting 
point, and which he referred to as “propositional plans” and “process plans” 
(1987a). This identical dichotomy was highlighted by Waters (2012) over two 
decades later, but he defined them as the “learning to communicate” and 
“communicating to learn” approaches. Although most ELT publishers have 
focussed on a learning to communicate approach in their coursebooks, in 
other words a product approach, developments in ELT methodology have 
also promoted a communicating to learn approach, in which the “product”, 
or the linguistic outcomes, are not pre-specified and instead develop more 
spontaneously due to the involvement of learners in classroom processes and 
activities. Examples of this approach include stronger forms of task-based lan- 
guage teaching, as well as Content and Language Integrated Learning (CLIL) 
instruction, and Dogme ELT (Waters, 2012, p. 443). 

Therefore, given the research over the years and these differing assump- 
tions held about language learning, it seems short-sighted to base a syllabus 
exclusively on either one type or another given the relative weaknesses of 
each. Breen, already in his article in 1987, postulates that process plans might 
be synthesised with propositional plans (1987b, p. 172). Clarke (1991, p. 25), 
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recognising the difficulties that a pure negotiated syllabus would create both 
for the teacher and the learners, makes reference to a “mixed” or “eclectic” 
syllabus where negotiation is used to adapt an existing product syllabus. 
Breen revisits this theme in his chapter published in 2001. Although he high- 
lights the diversity of syllabus types at the time, including the emergence of 
syllabuses based on lexical approaches including corpora studies, synthesised 
approaches to syllabus design that he had suggested in 1987 were not appar- 
ent. However, he predicts that because of the complexity of language learning, 
which involves not only knowledge about language but also communicative 
competence and many other broader aspects, the future of syllabus design 
would focus on what he describes as “multi-dimensional” syllabuses where 
elements from various syllabuses would be merged (2001, pp. 156-157). Long 
also refers to the “miscellany of syllabus types” that have emerged, amongst 
which are “hybrids” (Long, 2005b, p. 3). 

More recently, Wette (2011, p. 143) notes that Breen’s prediction has been 
correct and that blends of product and process-oriented syllabuses can now 
be found. Graves (2008, p. 161) also refers to other types of syllabuses, includ- 
ing the text-based syllabus, genre approaches to syllabus design and content- 
based syllabuses, and claims that we are now in a post-syllabus phase where 
“no one approach can be responsive to learners’ needs”. Instead of imposing 
an innovative approach to syllabus design, however, she states that the cur- 
rent models being used should be developed still further and this should be 
done by the individual classroom teachers, who in any case are the people best 
placed to be able to respond to the differing needs, expectations and desires 
of different groups of learners (Graves, 2008, p. 77). Littlewood (2009, p. 252) 
focuses on the “process-as-outcome” approach where the results of processes 
can be more easily observed and verified due to the important role issues con- 
cerning accountability and perceived value for money play in modern educa- 
tion. 

As far as the literature on syllabus design is concerned, therefore, the fo- 
cus on blends as an emerging approach has been highlighted by several re- 
searchers (Breen, 2001; Graves, 2008; Littlewood, 2009; Wette, 2011) because 
of the great complexity of the language learning process and the huge variety 
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of contexts in which it occurs. However, there seems to be no literature docu- 
menting classroom-based applications of an intentionally blended process- 
product approach to syllabus design, where elements of a negotiated syllabus 
(process approach) are deliberately introduced into a predominantly skills- 
based syllabus (product approach) to enhance the learning of those skills 
through opportunities for more practice. Consequently, the time seems ripe to 
implement such an approach in order to ascertain whether a blended ap- 
proach can really overcome many of the weaknesses that a purely product or 
process approach demonstrates. 


2.5 The Role of Practice in Language Skills Learning 


The ESAP syllabus that is the focus of this study already contained an empha- 
sis on practising and improving the skills when it was decided to redesign it 
following the findings from the data collection. From these findings, it became 
clear that a focus on the skills was still appropriate for the context. Although 
“there is no fully-fledged ‘skills-based approach’ to language teaching” (John- 
son, 2002, p. 190), a focus on skills has generally been classed as a product 
syllabus due to the inclusion of skills learning in both formal and functional 
syllabus types. It has been suggested that this has been particularly apparent 
in the area of ESP: “product-oriented approaches ... form the foundation of 
most courses based on commercial textbooks as well those that aim to develop 
skills and knowledge in English for academic or specific purposes” (Wette, 
2011, p. 137). 

The focus on language as a skill has an extremely wide literature, and it 
is beyond the scope of this section of the literature review to investigate the 
multiple aspects covered over the years, which range from general learning 
theories on skill acquisition without necessarily referring to language (Hold- 
ing, 1965) to detailed accounts of skill theory and language teaching (Johnson, 
1996). However, one aspect that has gained ground in cognitive approaches 
to skill acquisition in general is how practice can contribute to the automati- 
sation of skills, based on Anderson’s three-stage skill acquisition model of de- 
clarative knowledge leading to procedural knowledge to automatic output 
(2000). In most forms of skill acquisition, whether that be speaking an L2 or 


driving a car, learners are first presented with information about that skill, 
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and they then have to encode the skill to be able to perform the skill. This 
encoding, where all the actions and structures are examined step by step, is 
often demanding and error-strewn due to the cognitive load that remember- 
ing and sequencing the skill requires and as such is regarded as the declarative 
phase. The second phase of skill learning is when there is a shift from ardu- 
ously following this declarative information to incorporating it into proce- 
dural knowledge through practising the skill. In this proceduralisation stage, 
“the learner develops efficient procedures of performing the skill, often by 
skipping or compounding steps that were presented in the first stage” (D6- 
myei, 2009, p. 154). The advantage of reaching this procedural knowledge 
stage is that the learner can quickly use the rules learnt with a low error rate, 
but there is an inability to generalise that knowledge (DeKeyser, 2007b, p. 3). 
The third stage, known as automatisation, in its widest sense refers to the 
whole process from the initial engagement with the skill and its rule in its 
declarative state to “the final stage of fully spontaneous, effortless, fast and 
errorless use of that rule, often without being aware of it anymore” (DeKeyser, 
2007b, p. 3). In order to achieve automatisation in a skill, the skill requires 
practice, which has been noted by many over time (e.g. Ausubel et al., 1978; 
Johnson, 2002; DeKeyser, 2007b). Practice can be defined in many ways and in 
a very broad sense it may “simply involve using the skills that have been ac- 
quired, sometimes imperfectly” (Legge, 1986, p. 22). However, as far as prac- 
tice in the language classroom is concerned, DeKeyser (2007b, p. 8) maintains 
that practice refers to “specific activities in the second language engaged in 
systematically, deliberately, with the goal of developing knowledge of and 
skills in the second language”. Lightbown also provides a definition of prac- 
tice “as opportunities for meaningful language use (both receptive and pro- 
ductive) and for thoughtful, effortful practice of difficult linguistic features” 
(2000, p. 443). Moreover, Newell & Rosenbloom state: “Practice makes perfect: 
Correcting the over statement of a maxim: Almost always practice brings im- 
provement and more practice brings more improvement” (1981, p. 1). What 
can be seen from these three definitions is that if practice of a language skill is 
to have beneficial results, it needs to be undertaken purposefully and as reg- 


ularly as possible. 
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There has been much debate in the literature about how practice can be incor- 
porated into classroom work in order to contribute to language learning, and 
reviewing this extensive debate is again too vast for the scope of this section. 
However, Ortega, based on the interaction hypothesis literature (e.g. Long, 
1996; Swain, 2000) contends that “meaningful use of the L2, and particularly 
the meaningful productive use afforded during communicative interactional 
practice, drives acquisition” (2007, p. 180). She argues that to achieve this, 
practice in the classroom should follow three principles: it should be interac- 
tive, it should be meaningful and there should be a focus on task-essential 
forms (Ortega 2007, p. 180). She states that interactive practice between peers 
rather than with the teacher has been shown to provide more opportunities to 
talk for longer and “learner-learner interaction affords better opportunities for 
the expression of a wider variety of meanings and functions than is typically 
possible in teacher-fronted classrooms” (Ortega, 2007, p. 182). Her second 
principle of practice is that it should be meaningful, which she defines as be- 
ing the “cognitive engagement or involvement with the task, a quality that 
arises from the learners being personally committed to the practice event” 
(Ortega, 2007, p. 183). This cognitive aspect of meaningfulness has been linked 
to motivation theories in L2 learning (Dörnyei, 1994). The third principle she 
advocates is that the tasks learners are required to undertake in class are de- 
signed so that they “practice form-function connections in the L2 that the task 
makes particularly salient” (Ortega, 2007, p. 185). With this, she claims that 
often task-based practice can occur without a specific focus on form and so the 
tasks designed should exemplify language and communication in the target 
situation, which, as mentioned earlier in this literature review, has been advo- 
cated by others (Breen, 1987b; Long & Crookes, 1992). Consequently, to sum- 
marise, if practice of language skills in the classroom is to be beneficial in skills 
acquisition, learners need to be provided with opportunities to engage pre- 
dominantly in learner-learner interaction and the tasks in which they engage 
should provide target language practice and are meaningful to them. These 
last aspects would therefore imply their communicative and language needs 
would have to have been assessed and analysed for the tasks to be meaningful 


and relevant. 
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A final consideration about skills practice in the language classroom concerns 
the third stage of Anderson’s acquisition model; automatisation. In general 
skills acquisition theory, automatisation of a skill can take many thousands of 
hours but there will never be enough time in the L2 classroom to provide 
learners with the amount of practice that is needed to automatise a skill 
(DeKeyser, 2007c). However, DeKeyser argues that L2 skills practice in the 
classroom should not strive to achieve automatisation but rather should be 
seen “as a necessary but far from sufficient condition for the development of 
second language skills” (DeKeyser, 2007c, p. 292). Since “educators and edu- 
cational psychologists do not doubt the importance of practice” (DeKeyser, 
2007b, p. 5), and considering the evidence that has shown that meaningful, 
regular practice can lead to improvement in language skills proficiency, a syl- 
labus that has a focus on providing opportunities for target language skills 
practice would be beneficial especially in contexts where opportunities for 
practice are otherwise lacking, such as in situations where the language is 
used as a foreign language or in traditional educational contexts where using 
particularly the productive skills occurs infrequently. 

Given these considerations about the importance of practice in order to 
improve language skills, and the context in which the ESAP course is located, 
the intention to integrate as much skill practice as possible in the syllabus was 
clear. Furthermore, the use of a negotiated syllabus would allow for increased 
language practice as “it functions as a catalyst for the development of the 
learners’ communicative ability since, from the beginning, the negotiation is 


attempted in the target language” (Serrano-Sampedro, 2000, p. 109). 


2.6 Learner Autonomy 


The concept of learner autonomy, as has been discussed elsewhere (Prior 
2020), has been widespread in language education since the early 1980s. How- 
ever, there has been less focus on how a negotiated syllabus can contribute to 
learners achieving greater autonomy in their language learning. 

Briefly, learner autonomy has been described as “the ability to take charge of 
one’s own learning” and “to have, and to hold, the responsibility for all the 
decisions concerning all aspects of this learning” (Holec 1981, p. 3). This idea 
of autonomy originated from the practice of self-directed learning, often in 
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self-access centres, which is “learning in which the objectives, progress and 
evaluation of learning are determined by the learners themselves” (Benson, 
2001, p. 8). Despite the popularity of self-access centres with their materials 
and resources, there has not been any convincing evidence that self-directed 
learning alone can develop into learner autonomy and so the pursuit of 
learner autonomy moved away from the unrealistic assumption that it would 
develop spontaneously from self-directed learning and concentrated more on 
learner training (Holec, 1980). Holec focused on learners training themselves, 
but the practice of learner training has developed over the years and is no 
longer confined to self-directed learning and now tends to be incorporated 
into classroom learning (Benson, 2001). Learner training can take various 
forms, including language awareness training, which focuses on improving 
the learners’ knowledge about the nature of language and acquiring effective 
learning strategies (Dickinson, 1988). Learner training is important for all 
learners but “it is essential for those aiming at some level of autonomy” (Dick- 
inson, 1988, p. 46). 

The fact, therefore, that autonomy requires learners to take charge of their 
learning and take responsibility for it has been broadly accepted (Little, 1995; 
Cotterall, 2000; Benson, 2001), but this view can tend to focus on learners’ con- 
current language learning. Littlewood (1999) recognises that learner auton- 
omy is a goal that should be reached in order to benefit learners not only dur- 
ing their educational experiences learning languages but subsequent to that. 
Little also recognises how acquiring a degree of autonomy benefits learners 
both during and after their language educational experiences, as they can at- 
tain a degree of “pedagogical autonomy” that will be of an advantage in any 
learning context. However, he also states that “the whole point of developing 
learner autonomy is to enable learners to become autonomous users of their 
target language” and therefore exercise “communicative autonomy” (1995, p. 
176). If pedagogical autonomy is encouraged, learners will be able to practise 
it and make use of it directly during any classroom activities as well as outside 
the classroom while engaged in their language learning, even if they are una- 
ble to exercise communicative autonomy because their language proficiency 
level is too low. However, as learners increase their language proficiency, ped- 


agogical autonomy and communicative autonomy will then begin to interact 
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and learners will be more confident when using the target language in con- 
texts away from the classroom than learners who have not practised learner 
autonomy. Little (1995) makes it clear that confidence in using the language is 
crucial for successful language learning. 

Moreover, fostering learner autonomy can be seen as being even more 
desirable now than in the past, given how English has become so pervasive in 
so many contexts, whether they be cultural, social, educational or technologi- 
cal. The use and learning of English in the globalised world are ever-changing 
due to the rapid growth of communication technologies and the omnipresence 
of the internet, which has provided language learners with an infinite supply 
of input, stimulus and communicative opportunities. Waters, in his review of 
ELT methodology, states that “the increasing ubiquity of web-based language 
teaching and learning resources has the potential to redistribute the balance 
between teacher-led and learner-based instruction” (2012, p. 448). As a conse- 
quence, learners have more opportunities than ever to work more inde- 
pendently, whether that be explicitly in language learning environments, or 
in their every-day lives. Moreover, the assumption that the English we use 
today can be represented by a homogenous and monolingual culture is clearly 
irrational given the global status that English now enjoys. Illés (2012) refers 
specifically to the established but unrepresentative native-speaker models 
that have traditionally been used, and to a certain extent are still used, in ELT 
but which can no longer be considered appropriate for 21st century English- 
language learners who have to operate in a globalised world where English is 
used so prevalently. She therefore believes that “the task of language educa- 
tion is ... to help learners develop self-reliance and autonomy, which will en- 
able them to communicate successfully in international settings” (Illés, 2012, 
p. 506). These developments make it fundamental that there is a focus on ways 
to foster learner autonomy in any context where a syllabus, especially a 
learner-centred syllabus, is being designed and implemented in English-lan- 
guage courses. 

Cotterall agrees that learner autonomy should be integrated into class- 
room practice and she focuses particularly on how teachers can help foster 


learner autonomy in their courses. She states that 
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Language courses which aim to promote learner autonomy will incorporate means 
of transferring responsibility for aspects of the language learning process (such as 
setting goals, selecting learning strategies, and evaluating progress) from the 
teacher to the learner. (Cotterall, 2000, pp. 109-110) 


This approach, therefore, regards the learner as a decision-maker who has a 
certain amount of control over aspects of the learning process, and Benson 
states this explicitly: “the key factor in the development of autonomy is the 
opportunity for students to make decisions regarding their learning within a 
collaborative and supportive environment” (2001, p. 151). Typically these de- 
cisions concern the planning of classroom activities and the evaluation of their 
outcomes but Cotterall asserts the real challenge for course designers is “to 
find ways of supporting the transfer of responsibility for decision-making 
about learning from teacher to learner” (2000, p. 110). This approach implies 
that the teacher and the learners should engage in activities that will allow the 
learners to express their needs and interests and provide opportunities for re- 
flective feedback, both from the teacher and the learners. In her discussion of 
the context in which her study occurred, this is effectively what happened as 
her learners, all low-level adult learners, were initially asked to set goals for 
the course, were encouraged to keep learner journals, had regular interviews 
with the teachers and engaged in ongoing reflection both with their teachers 
and their peers (Cotterall, 2000). Moreover, many tasks in class were devel- 
oped based on the individual learner’s goals and future communication situ- 
ations so that “rather than having to create links between pedagogic tasks and 
their own needs, learners instead practised tasks associated with their target 
situations, and received feedback on their performance” (Cotterall, 2000, p. 
114). In the evaluation of this course, the inclusion of tasks related to learners’ 
goals, resulted in “an unprecedented level of motivation” (Cotterall, 2000, p. 
115). This increase in motivation has also been observed by Little, who argues 
that learners who take responsibility for their own learning not only have a 
greater likelihood of reaching their learning goals, but they also tend to “main- 
tain a positive attitude to learning in the future” (1995, p. 176). 

In Cotterall’s study, although not once does she refer to a negotiated or 
process syllabus, the approach she describes is almost identical to the models 
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of the negotiated syllabus that were presented in Section 2.5.2. However, she 
does refer to the process that was undertaken and states that it 


presented the learners with a means of meeting their own needs. By making the 
language learning process salient, the course helped learners understand and man- 
age their learning in a way which contributed to their performance in specific lan- 


guage tasks. (Cotterall, 2000, p. 115) 


Other studies have also implemented approaches where learners have taken 
responsibility for decision-making regarding their learning (Hall & Kenny, 
1988; Karlsson et al., 1997; Dam, 1995) and the syllabus models used are very 
similar to the negotiated syllabus. However, also in these cases the terms “ne- 
gotiated” or “process” syllabus are never used and so a clear link between 
promoting learner autonomy within a negotiated syllabus was not made in 
these accounts. A clear link is established by Bloor & Bloor, however, in their 
paper “Syllabus Negotiation: the basis of learner autonomy”. They describe 
syllabus negotiation as an “approach to helping students arrive at the position 
of being able to understand and articulate their language learning objectives” 
(1988, p. 62) and they regard syllabus negotiation as a crucial way to encour- 
age learners to take responsibility for their own learning, the fundamental 


premise of learner autonomy: 


Whether it be with an individual in a self access programme or with a group of 
students in a conventionally taught course, negotiating the syllabus is the first step 


towards full responsibility. (Bloor & Bloor, 1988, p. 65) 


In their account of a negotiated syllabus for an academic writing course at the 
University of Warwick, they identify the limits that the context presented, 
compared to the self-access courses to which they also refer. In the case of the 
self-access courses, they were able to negotiate the “broad objectives” of the 
course due to the individual nature of the courses, whereas with the taught 
courses they identify that it is the “details” that can be negotiated (Bloor & 
Bloor, 1988, p. 70). This account and its approach follows a similar approach 
to that presented by Breen & Littlejohn (2000c) in their negotiation cycle and 
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the curriculum pyramid as discussed in Section 2.5.2, and demonstrates that 
promoting learner autonomy often follows a very similar framework to that 
of the negotiated syllabus. 

Promoting learner autonomy, therefore, is a central tenet in the study that 
is the focus of this volume and as such, plays a significant role in the syllabus 
that has been designed and implemented for the ESAP course that is the sub- 
ject of this research. Providing the students with a degree of autonomy in their 
language learning is crucial because, as noted before, they will not have a 
teacher to accompany them throughout their English-learning experiences. 
Further, a constraint of my particular context is that the ESAP course in ques- 
tion only includes 30 contact hours in the classroom. Because there is such a 
short time available for this course, providing the students with opportunities 
to exercise a certain degree of autonomy in their learning English is a means 
to optimise the time that is available. Cotterall identifies this aspect as a reason 
for promoting learner autonomy because this approach “freed [her] from the 
unrealistic challenge of attempting to meet 20 different learners’ needs within 
a 10-hour course” (2000, p. 115). Moreover, by promoting learner autonomy, 
the students will not only be able to take responsibility for some of the deci- 
sions linked to their language learning, but they will also be engaging in com- 
municative activities that should benefit their language learning not only 
when following this course but also in any future language learning opportu- 
nities that they encounter. Therefore, due to these reasons, and the pragmatic 
underpinnings of this study, promoting learner autonomy is considered key 
in the design of this syllabus. 


3. Action Research 


3.1 Introduction 


In this chapter, action research (AR), the research tradition used in the study, 
is presented. A chapter solely devoted to action research was deemed neces- 
sary because this study not only used AR as its research framework, but it was 
the use of multiple AR cycles that informed many of the actions and directions 
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taken in the study. Therefore, this study developed in the way it did precisely 


because it was an action research study. 


3.2 Historical Overview and Use in Education 


Action research emerged as a research tradition in the second half of the twen- 
tieth century “as a reaction to scientific, experimental and quantitative para- 
digms” (Burns, 2005b, p. 57) and as such is grounded in constructivism. It is 
an approach used to explore aspects in a social context and improve the prac- 
tices within those contexts and is characterised by its dual focus on action and 
research. The action element comprises participants in a process of deliberate 
intervention within a social context where there are problems or aspects have 
been problematised in order to bring about changes and improvements. The 
research element involves the systematic observation and analysis of these in- 
terventions and the changes that result, and reflection on the implications that 
these results create. 

Action research is prevalent in a large variety of disciplinary fields, from 
health care to business and management (Burns, 2005b). It has also now be- 
come a much more common research tradition in applied linguistics and ed- 
ucation due to educational research having become methodologically diverse 
over the past few decades (Burns, 2010). 

The developments in the use of AR in educational contexts have led to a vari- 
ety of definitions of AR. One of the simplest is possibly from Elliott who de- 
scribes it as “the study of a social situation with a view to improving the qual- 
ity of action within it” (1991, p. 69, as cited in Altrichter et al, 1993, p. 4). Carr 
& Kemmis take a more comprehensive view, taking into consideration the 


critical aspect of AR linked to practice when they describe it as: 


self-reflective inquiry, undertaken by participants in social situations in order to 
improve the rationality and justice of their own practices, their understanding of 
these practices, and the situations in which these practices are carried out. (Carr & 
Kemmis, 1986, p. 220) 


The idea that AR is a form of reflective practice that focuses specifically on solving 
problems in the context has been suggested by various researchers (Burns, 1999, 
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p. 30; Levin & Greenwood, 2001, p. 105; Coghlan & Brannick, 2005, p. 4; Burgess 
et al., 2006, p. 60; Costello, 2011, p. 7; Bradbury et al., 2019, p. 7). However, Burns 
(2010, p. 2) in her later book, when referring specifically to AR that is used in ELT, 
defines AR as “a self-reflective, critical, and systematic approach to exploring your 
own teaching context” where teachers problematise a situation and then attempt 
to find approaches to improve that situation. Action research conducted in edu- 
cational contexts therefore, does not have to originate necessarily from problems 
within those contexts, but it always aims to effect change that will lead to improve- 
ment. 

Kemmis further underlines the focus on the teacher’s own context in his 
definition, but goes further, stating that the principal characteristic of AR is 
that “it always is a practice-changing practice” (2009, p. 464 original italics), em- 
phasising how the engagement with AR itself brings about change within 
teachers’ practice. In fact the constructivist influence on AR has led it to be 
described as “a response to conventional teaching” (Kayaoglu, 2014, p. 145), 
given constructivism’s focus on the individual and learner activity in the class- 
room (Biesta, 2013). Consequently AR, because of its focus on reflective and 
self-reflective practice, can lead to modifications in classroom practice that 
might even oppose the prevailing orthodoxy. 

Another central aspect of action research is its focus on the researcher(s) being 
integrated into the study rather than viewing it from outside. It was particu- 
larly this factor that made it a participative, exploratory research tradition, 
and made it part of the constructivist movement that developed over the 
twentieth century (Burns, 2005a). Moreover, the focus on the researcher him 
or herself being involved in a “systematic process of enquiry arising from their 
own practical concerns” (Burns, 2005a, p. 241), distinguishes AR from other 
forms of applied research, which tend to concentrate on the application of 
general theoretical issues to a context that is not related to the researcher’s 
practice. This focus on the researcher’s own practice has created a branch of 
AR known as insider action research, where the researcher is a permanent 
member of the professional or educational context which s/he is studying. 
Coghlan (2007) states that the main characteristics of insider AR are that it is 
interventionist, so it always aims to change aspects of the researcher’s own 


context, and that it generates “actionable knowledge, which can be defined as 
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knowledge that is useful to both the academic and the practitioner communi- 
ties” (Coghlan, 2007, p. 296). Kemmis (2009, p. 468) expands this view still 
further, stating that AR not only allows practitioners to be researchers but it 
also gives “practitioners intellectual and moral control over their practice”. As 
such, it is a process that invites reflection about the practice, which clearly 
affects how teachers work and relate to what they are doing, but it can also be 
self-transformative, where teachers actually change the way they approach 
their lives (Kemmis, 2009; Herr & Anderson, 2015). 

To a certain extent, action research has been considered to be a collabora- 

tive endeavour undertaken with other teacher-researchers rather than an in- 
dividual pursuit (Kemmis & McTaggart, 1988, p. 6; Burns, 1999, p. 1) although 
Nunan disagrees, stating that collaboration cannot “be seen as a defining char- 
acteristic of action research” (1992b, p. 18). Consequently, action research is 
undoubtedly practice-based, but can be undertaken both in collaboration with 
other teacher-researchers or alone. In this study, I worked alone. 
Action research as we mainly understand it today began its development in 
the 1980s, again in Australia by Kemmis and colleagues, who reintroduced 
Lewin’s cyclical form and need for self-reflection and proposed a new model 
with four clear “moments”, being Plan — Action - Observation — Reflection 
(Kemmis & McTaggart, 1988) as illustrated in Figure 3.1. 
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Fig. 3.1 — The action research cycle(s 


This focus on practice and improvement should have made AR appealing for 
ELT practitioners, yet it was only in the late 1980s that teachers really started 
to engage in action research projects in ELT (van Lier, 1988). In fact, many AR 
studies in ELT around that time were focused on curriculum evaluation due 


to the considerable influence of communicative language teaching on the 
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changing role of the teacher and how this role could become more interven- 
tionist (Burns, 2005a). At the end of the 1980s, Nunan (1988b) had advanced 
the idea of the “learner-centred curriculum” which tended to “place the onus 
for course design and curriculum development on classroom teachers” (Burns 
& McPherson, 2017, p. 107) and thus created a need for more opportunities for 
professional development. AR studies in ELT also focused on enhancing pro- 
fessional development in teacher education as well as on other aspects, such 
as finding solutions to particular problems in specific teaching situations. As 
a consequence, AR became “a byword for integrating a research orientation 
into the work of practitioners” (Burns, 2005a, p. 245). However, Borg (2010) 
contends that it is still relatively unusual for English-language teachers to en- 
gage in any research, not just action research, often because these teachers ei- 
ther lack research knowledge and skills or they tend to regard research as be- 
ing detached from their classroom practice. Therefore, the study that is the 
focus of this volume is, to a certain extent, a model of how ELT research is 
described by various researchers. Not only does it focus on research con- 
ducted in a real-life context but it also aims to redesign a syllabus to improve 
classroom practice, aspects that are all seemingly unusual. 

In education in general, and also in ELT, therefore, the essential element 
of AR is that it aims to improve the quality of the practice in the classroom, 
thus benefitting both the learners and the teachers involved in the process as 
well as colleagues and other practitioners in analogous contexts through dis- 
semination of the results of AR projects. AR brings theory and practice to- 
gether both to raise awareness of a (pedagogical) situation and its limitations 
and/or problems and to improve that situation through implementation of an 
action. This process is often undertaken by the teacher involved in the situa- 
tion and accordingly introduces the figure of the teacher-researcher who is 
best placed to act as “an active catalyst for change in teaching practices, course 
design and problem solving” (Fareh & Saeed, 2011, p. 155). 


3.3 Cycles and Types of Action Research 


Another essential feature of AR, as briefly mentioned in the previous section, 
is that it is not a linear process but cyclical, which Edge refers to as a “spiral 
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of increasingly aware experience” (2001, p. 3). Typically in AR studies, a se- 
quence of stages occurs, which Edge characterises as: 


Action in the sense that one is in the middle of action 

Observation of what is happening, leading to a more specific focus of interest 
Reflection on the focus which has been identified, leading to 

Planning that forms the basis for future 

Action to improve the situation, supported by 


Observation in order to evaluate the changes made (Edge, 2001, p. 3, original italics) 


These sequences that typify AR are very similar to the widely accepted model 
of “plan-act-observe-reflect” as presented by Kemmis & McTaggart (1988, 
p. 10, see Figure 3.1) and which they describe as: 


develop a plan of critically informed action to improve what is already happening; 
act to implement the plan; 

observe the effects of the critically informed action in the context in which it occurs; 
reflect on these effects as the basis for further planning, subsequent critically action 


and so on, through a succession of stages. 


Both these models focus on AR’s objective to change and improve the situa- 
tion under study and this emphasis on change is also highlighted by 
Denscombe, who suggests that AR only has four characteristics: its practical 
nature, its focus on change, the involvement of a cyclical process, and its con- 
cern with participation (2007, p. 123). 

In the case of the study of this volume, all the main characteristics identi- 
fied by Denscombe are present: it is a practice-oriented study where a new 
syllabus is implemented in order to change and improve the existing syllabus, 
and within the cyclical process, evaluation stages are implemented to ascer- 
tain to what extent the new syllabus is benefitting the participants concerned. 
Consequently, the participants in this study — the students — have been able to 
participate actively in the evaluation of the effectiveness of the new syllabus, 
as well as being able to participate actively in the shaping of aspects of the 
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syllabus itself, due to the syllabus’s focus on negotiation with those involved 
in the learning process. 

The kind of action research described above and used in this study is one 
of the three distinct types of AR identified by Carr & Kemmis (1986), who split 
AR projects into technical AR, practical AR and critical AR. Technical AR fo- 
cuses on the outcomes of the teacher or researcher’s practice and is a one-way 
relationship with the other participants. Practical AR, on the other hand, alt- 
hough directed by the practitioner, involves the other participants who have 
a voice in the process. In this type of AR, not only does the practitioner listen 
to these voices, but s/he also “remains open to ... the consequences that these 
others experience as a result of the practice” (Kemmis, 2009, p. 470). Critical 
AR is generally even more participatory, where everyone contributes to the 
research collectively (Kemmis, 2009). 

Consequently, given these distinctions, as well as the fact that the 
worldview informing this study is pragmatism, as will be discussed in the 
following chapter, my study can be seen as a practical action research project, 
which is directed by me as the teacher-researcher. However, the other partic- 
ipants — the students and to a lesser extent the lecturers using EMI — have a 
voice due to the AR cycles which include regular data collection stages in or- 
der to provide evaluation throughout the entire project. 


3.4 Effect on the Researcher/Teacher 


So far this chapter has only concentrated on the history and theory of AR in 
educational contexts and particularly in ELT, and the purposes and goals of 
implementing an action AR study. However, studies have suggested that 
there are many benefits to implementing AR in educational contexts (Burns, 
2009; Burns, 2013; Edwards & Burns, 2016), and that engaging in AR studies 
also has many positive outcomes on the main actor(s) of the study, in other 
words the teacher. These positive outcomes range from feeling “able to make 
confident, evidence-based decisions about their teaching” (Edwards & Burns, 
2016, p. 10) to it being generally “transformative” (Borg, 2010) and even “self- 
transformative” (Kemmis, 2009) for the teachers involved. Zeichner (2003), 
however, cautions that many reports are anecdotal and the contexts are so di- 


verse that many of these accounts are unreliable. 


39 


Integrating Classroom-Based Negotiation Into a Syllabus 


Despite these concerns, there is evidence that AR’s focus on self- 
reflective and critical practices has led many teachers to feel empowered from 
participation in AR studies. Zeichner (2003) reports that under certain condi- 
tions, AR can provide teachers with more self-confidence in their ability to 
promote student learning and to become more proactive when dealing with 
difficult issues that arise in their teaching. Burns (1999) also referred to this 
feeling of empowerment and presented many positive comments from teach- 
ers who were engaged in a large AR study. These ranged from being able to 
engage more fully with their classroom practice to increased self-awareness 
and personal insight. One even highlighted how they had experienced both 
professional and personal growth: 


I felt a degree of personal satisfaction once I had collected the data and completed 
the write-up — a feeling that I had challenged myself and was able to meet the chal- 


lenge to a certain extent. (Burns, 1999, p. 14) 


Empowerment can also originate from AR’s democratic principles, since “it 
invests the ownership for changes in curriculum practice in the teachers and 
learners who conduct the research” (Burns, 2010, p. 10). 

The results of these democratic principles have been shown elsewhere. 
Zeichner (2003) presents evidence that teachers who have had positive expe- 
riences conducting AR studies have attained successful outcomes, such as 
moving gradually towards more learner-centred instruction, predominantly 
through engaging with the learners more frequently and eliciting feedback 
from them. A similar finding was evident in a study by Edwards & Burns 
(2016, p. 11), who found that teachers were able “to establish more open, col- 
laborative approaches to teaching” due to becoming more reflective in their 
teaching after having engaged in AR. Zeichner (2003, p. 309) also found in his 
study that “many of the teachers reported improvements in pupil attitudes, 
involvement, behavior, and/or learning as a direct result of specific actions 
taken in their research”. Improvements in students’ attitudes, in particular re- 
ferring to attendance, were also observed in my study, and further details and 
discussion concerning this aspect can be found in Chapter 7 in the discussion 
of the implementation of the blended syllabus. 
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3.5 Criticism of Action Research 


Despite the reported benefits of AR, there has also been much debate about 
and criticism of AR mainly because it combines practice and theory and, as 
such, is seen by some to be too “subjective” (Burns, 2010, p. 95) or lack rigour 
(Bradbury & Reason, 2001; Burns, 2005b). K. Richards (2003, p. 26) notes that 
particularly due to its recent popularity, many projects have been classed as 
AR although they have not stood up to the rigour qualitative research re- 
quires. However, Block (2000, p. 138) takes an opposite view and criticises AR 
because “the entire enterprise is strong in theory but very difficult to carry out 
in practice”. Action research, and particularly Kemmis & McTaggart’s plan- 
act-observe-reflect sequence, has also been criticised for being overly prescrip- 
tive as the spiralling stages might be considered too inflexible a framework, 
which could limit the freedom of teachers engaged in action research (Burns, 
2010, p. 8). However, Burns (2005a, p. 250) also argues that “one of the strong- 
est features of action research that can contribute to enhancing rigor is its iter- 
ative, or cyclical, nature” because the cycles provide opportunities for data 
collection, observation and reflection so that findings can be tested and then 
developed, enhanced or even modified. 

AR also tends to be criticised because it is often implemented by — or at 
least involves — the practitioner, i.e. the teacher, who can be regarded as being 
inexpert in research. Burns (2005a) refers to criticism as far back as the early 
1980s where doubts were expressed about the ability of teachers to engage in 
research. Johnston (1994) claimed that the action research process is unnatural 
to teachers, and if it occurs it tends not to be initiated by the teachers them- 
selves but by an external source. Borg (2010, p. 404) makes reference to the fact 
that teacher research in general can often seem to focus on descriptive ac- 
counts rather than analysis. Much of this criticism concerning the allegedly 
poor quality of AR undertaken by teachers often stems from conventional sci- 
entific concepts of research, such as studies should be large-scale, replicable 
and quantitative. This view demonstrates a fundamental misunderstanding of 
one of AR’s central goals, which is to provide understanding about a local 
context, which is often extremely specific and small. However, in order to sup- 


port teachers who might be less familiar with research, Burns (2005b) suggests 
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teachers work collaboratively with a researcher to improve the reliability of 
their findings. This might not always be feasible, so she also suggests that 
teachers should work within a clear methodological framework that defines 
the procedures undertaken explicitly in order to strengthen the study. 

Further criticism, again mainly originating from conventional interpreta- 
tions of how research should be undertaken, refers to the small-scale nature 
or the specific contexts that tend to typify action research studies. Zeichner 
refers to criticism directed at the usefulness of knowledge generated by teach- 
ers due to their specific teaching and learning contexts (2001) and Burns 
(2005b, p. 67) refers to how AR can lack “external validity” due to the small- 
scale nature of many studies and the impossibility for them to be generalisa- 
ble. However, she also recognises that because action research studies are con- 
cerned with practical solutions, they “might have resonance for other practi- 
tioners in comparable situations” (Burns, 2005b, p. 67) and as such, external 
validity is not necessarily a goal for AR. 

Although AR has been criticised for involving teachers, the relative rarity 
of teachers engaging in action research has also been noted. Rainey (2000) in her 
study found that over 75% of the 228 teachers she surveyed said they had never 
even heard of action research and Dérnyei (2007, p. 191) states “there is one big 
problem with action research: there is too little of it”. There are predominantly 
three factors that contribute to this: the first is a lack of encouragement from the 
teacher’s institution or superior(s) and a resultant lack of motivation to do re- 
search (Burns, 2005b; Borg, 2010), the second, and most commonly cited reason, 
is a lack of time (Rainey, 2000; Borg, 2010) and the third is, as discussed, teachers’ 
general lack of research expertise (Dérnyei, 2007, p. 192). 


3.6 Rationale for Using Action Research in This Study 


As it has been stated in this chapter, AR aims to improve the quality of the 
practice in the classroom, thus benefitting both the learners and the teachers 
involved in the process. AR is therefore an appropriate tradition to use for this 
study because its objective is coherent with the aim of this study, which is to 
improve the quality of the teaching and learning practices in the ESAP courses 
I teach for economics students at the Faculty. Further, given the emphasis on 
practice that this study has, and the general set of beliefs and principles that I 
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hold as a teacher and a pragmatist, the use of AR with its focus on improving 
the quality of practice through practical action is consistent with this general 
worldview. Moreover, the AR process is often undertaken by the teacher in- 
volved in the situation and therefore s/he is best placed to enact change in 
his/her teaching practices. As I am the teacher of the ESP course in question, I 
am therefore the person who is best placed to undertake the process to provide 
the students with a course that is as relevant and useful as possible. 


3.7 Measures to Counter Criticism of Action Research 


In view of the criticism directed at AR, I will address the various issues and 
describe the action taken for each one. 

Regarding the concerns about rigour, the research methodology used in 
this study was carefully chosen and implemented taking into consideration 
the context, the timespan and the participants involved. The data collection 
and analysis followed a convergent parallel mixed methods design, and ap- 
plied a clear framework for data collection and analysis. This framework, us- 
ing two recognised models from quantitative and qualitative research that 
were adapted for the specific context of the study, was flexible enough to ac- 
commodate the multiple AR cycles but was designed to guarantee a precise 
structure for the quantitative and qualitative data collection and analysis also 
given the numerous phases that characterised the study. 

Despite Block’s assertion (2000, p. 138) that AR projects are difficult to 
conduct from a practical perspective, this problem was never experienced in 
this study. Because the context was the ESAP course I was teaching, because I 
had learned the skills needed to conduct the research through the resources 
provided through the PhD programme I had completed, and through the re- 
sources available to me in my work context, and because of my twenty-plus 
years of experience teaching and researching ESP at the tertiary level, the prac- 
tical issues that did arise were able to be resolved. In fact the two main prac- 
tical problems that affected the implementation of the redesigned syllabus — 
the lack of interest of the students in the negotiated syllabus in AR Cycle 2 and 
their infrequent and poor attendance in the PPE course in AR Cycle 3 — were 
able to be resolved also because this was an AR project with multiple cycles. 
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With respect to the criticism about AR precisely due to the use of cycles as 
mentioned in Burns (2010), the cycles themselves accommodated the iterative 
nature that evolved with the data collection, observation and reflection. With- 
out the flexibility within these cycles, the coding of the interviews would not 
have been as comprehensive as they became, and the opportunities to revisit 
the data (both quantitative and qualitative) in the light of the various versions 
of the syllabus used would not have been possible. 

Although it is understandable why criticism about AR projects is often 
directed at the lack of expertise shown by the teacher(s) who may be involved, 
the AR study which is the focus of this volume was a study conducted for a 
PhD programme and as mentioned, the programme provided an ample 
amount of training in research methodology. Further, my considerable expe- 
rience as a classroom practitioner and researcher working as an ESAP lecturer 
at a university allowed me to use the skills I already possessed and this study 
afforded me with the opportunities to develop them still further. 

It is also true that often AR studies are concerned with small-scale con- 
texts, which was most definitely the case in my study. However, Burns (2010) 
acknowledges that many action research studies are concerned with practical 
solutions, which was the case in my context, and so Iam convinced that alt- 
hough my context is specific, the situation is familiar to many ESP lecturers 
working in higher education institutions in Europe and elsewhere. Moreover, 
in order to provide as much resonance as possible, this study was conceived 
as a practical AR project where the data collection and analysis phases have 
been described in as much detail and as clearly as possible so that they can be 


easily replicated in analogous situations. 


3.8 Practical Application of the AR Cycles in the Study 


The AR cycles were organised and implemented in this study following the 
plan-act-observe-reflect sequence that was devised by Kemmis & McTaggart’s 
(1988) and illustrated in Figure 3.1. There were three cycles in total spanning 
three academic years (from May 2014 to July 2016). Figure 3.2 below illustrates 
the three cycles and their timeframe and to ensure clarity at this stage and for 
ease of reference hereafter, the cycles are referred to as AR Cycle 1, AR Cycle 
2 and AR Cycle 3. Each stage of each cycle will therefore be referred to with 
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these same numbers, so Action 1 refers to the action stage in AR Cycle 1 and 
Observe 3 refers to the observation stage in AR Cycle 3 and so on. 


PLAN 


Identify problem 
Re 
ACTION REFLECT 
Data collection QUANTITATIVE 
QUANTITATIVE: Questionnaire TPA merge 


QUALITATIVE: Semi-structured QUALITATIVE 
interviews 


May-September 2014 


OBSERVE 


Data analysis 


PLAN 


How to redesign 
syllabus based on 
data analysis 


ACTION X REFLECT 
Implementation of 


October 2014-July 2015 


redesigned syllabus Data analysis 
(Syllabus 1) 

OBSERVE 
Data collection 

QUANTITATIVE: 

Questionnaire TPB/ 
End-of-course surveys 
How to integrate 
findings from data 


collection to modified 
syllabus 


ACTION Ñ A REFLECT 
Implementation of 
modified syllabus Data analysis 
(Syllabus 2) 


Data collection 
QUANTITATIVE: 
Questionnaire TPC / 
End-of-course surveys/ 
Negotiated portfolio 
worksheet 


October2015-July 2016 


Fig. 3.2 — Action research cycles during the study. Note: TPA = student target population A; 
TPB = student target population B; TPC = student target population C. 
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3.8.1 Cycle 1 


The first cycle, which took place over a four-month period (May-September 
2014), comprised data collection where data were collected using a mixed 
methods approach. In Plan 1, the context was reflected upon and problema- 
tised. This resulted in the conclusion that the current syllabus for the ESAP 
courses I was teaching was not as learner-centred as I felt it could be and so 
there was the question as to whether it was adequately addressing the needs 
of the students. In this planning stage, the first two research questions of the 
study were formulated, which concerned which language skills the students 
needed to study at the Faculty of Economics and therefore how these would 
be reflected in the ESAP syllabus. Although these two questions were formally 
verbalised in AR Cycle 1, they had been in gestation for some time before the 
study had begun. It has been suggested that in insider action research studies, 
where the researcher investigates their own context, as is the case in this study, 
“there is not a clear beginning, or, for that matter, ending of the research” 
(Herr & Anderson, 2015, p. 92). In these cases, AR research questions are often 
formalised even some time after the situation under investigation has been 
identified. Therefore, for the purposes of this volume, the first two research 
questions of this study have been placed in Plan 1 of AR Cycle 1, and in fact 
were formalised in their current state at that time. However, in reality, the 
general questions had been conceived some time before the AR study was for- 
mally initiated. 

Once these questions had been formulated, the data collection methods 
and the respondents were identified. In Action 1, the data collection occurred, 
and the different sources used to collect data were approached. Qualitative 
data were collected from the lectures who use English as a medium of instruc- 
tion in the form of semi-structured interviews, and a questionnaire was ad- 
ministered to the economics students. In Observe 1, the data were analysed 
and in Reflect 1, the quantitative and qualitative data were merged, following 
the clear process of a mixed methods approach and as discussed further in 
Chapter 4. All further discussion about data collection and analysis is con- 
tained in Chapters 4 and 5. 
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3.8.2 Cycle 2 


AR Cycle 2 took place over the following academic year and the ESAP course 
in question was that of the Economics and Management (E&M) students (Feb- 
ruary 2015-July 2015), and the results from the data collection and analysis 
Action 1 and Observe 1 were used to plan an alternative approach to the syl- 
labus. It was at this point that the third and fourth research questions were 
formulated, which considered how the product aspects of the ESAP syllabus 
could benefit from being integrated by a process approach and which ele- 
ments of the syllabus could therefore be negotiated with the students. In Plan 
2, an approach using a blend of a product and process approach to syllabus 
design was devised to provide the students with more choice in the planning 
of their course and in Action 2, the new approach to the syllabus was imple- 
mented. Given the importance of evaluation in AR studies, the fifth research 
question was also formulated at this time, which focussed on the evaluation 
methods that could be devised to ascertain the effectiveness of this proposed 
syllabus for each individual learner. 

Once the syllabus had been implemented, Observe 2 was used to collect 
further data from the students, by conducting the same questionnaire that had 
been sent in AR cycle 1 in order to start collecting longitudinal results, as well 
as an end-of-course survey to measure students’ attitudes towards the new 
syllabus that had been implemented in AR Cycle 2, thus providing a means to 
conduct an evaluation and provide answers to research question 5. This stage 
was particularly important due to the fact that very few students had engaged 
with the new syllabus and it was important to understand why this had been 
the case and what could be done to enhance future applications of the sylla- 
bus. Burns, when discussing the AR cycles states that “often, the results of 
changes are unpredictable and reveal new or unexpected avenues for further 
action, which is then observed and documented further” (2009, p. 290). In fact, 
without the AR cycles, although the unexpected lack of interest from the stu- 
dents would have been observed and then reflected upon, there would have 
been no scope for intervention. However, with the multiple cycles of this 
study, action was able to be taken and in fact Action 3 was subsequently char- 
acterised by “critically informed action” (Kemmis & McTaggart ,1988, p. 10). 
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3.8.3 Cycle 3 


AR Cycle 3, which lasted the entire 2015/2016 academic year comprised the 
two ESAP courses: the one for the Politics, Philosophy and Economics (PPE) 
students in the first semester between October 2015 and January 2016 and the 
one for the E&M students in the second semester between February and June 
2016. A slight change was made in the implementation of the syllabus in the 
second semester with the E&M students to correct a problem regarding at- 
tendance that had been identified in the first semester. Therefore, as this prob- 
lem concerned a practical obstacle to the implementation of the syllabus, yet 
the syllabus itself remained unchanged, these two implementations are re- 
garded as one AR cycle in this study. 

Plan 3 consisted in redesigning the syllabus to mitigate the students’ lack 
of uptake of the first syllabus and was enacted in Action 3 over the first se- 
mester and then again in the second semester. During Observe 3, further data 
were collected from the students: the third online questionnaire was con- 
ducted to provide further longitudinal results in May 2016, and end-of-course 
surveys were completed by the PPE students in January 2016 and E&M stu- 
dents in June 2016 to provide information about the revised version of the 
syllabus. Data were also collected in class as field notes and from the work- 
sheets that students had used in class. 

In the first Observe 3 phase, as mentioned above, a problem concerning 
attendance was noted. The fact that the course did not require compulsory 
attendance was already a given in the context, but the irregular attendance 
that was the norm for the course had not been considered when the revised 
syllabus was introduced. Therefore, action was taken in the second semester 
course with the E&M students to resolve this unexpected problem, which will 
be discussed more in depth in Chapter 7. This problem, as already mentioned, 
was able to be identified and solved not only because action research was used 
as the research tradition in this study but also because multiple cycles were 
enacted. Consequently, this particular phase of the study exemplified the 
“driving purpose” of AR, which is “to bridge the gap between the ideal (the 
most effective ways of doing things) and the real (the actual ways of doing 
things)” (Burns, 2009, p. 291). 


48 


Research Methodology 


The final phase of this cycle, Reflect 3, was used to analyse the data from the 
students collected during the Observe 3 phase. The findings from this phase 
contributed to the most recent version of the syllabus. This syllabus has an 
almost identical form to the modified syllabus that was used in AR Cycle 3 in 
the second semester of 2015/2016, in that the blended approach used is the 
same, the main source materials remain largely unchanged and the constraints 
that are discussed in Section 7.3.1 are still valid. However, the contents based 
on those materials, the distribution of language work and some of the specific 
topics have changed since the study was undertaken. These changes are due 
to the focus on negotiation and the changing needs and interests of the stu- 
dents in each course each year, as well as the move to online teaching in March 
2020 due to the lockdown imposed by the COVID-19 pandemic. At the time 
of writing (the 2022/2023 academic year), the course has now resumed face- 
to-face, but the content will still be negotiated with the students and thus the 


course will again undergo changes compared to previous years. 


4. Research Methodology 


In this chapter the research methodology that was used in this project will be 
presented and will mainly focus on the approach to research used for the study, 
and then the research design and tradition used. The data collection methods 
that were used during the needs assessment stage of the study and during the 


AR cycles will be presented, as well as my approach to data analysis. 


4.1 Philosophical Framework 


Philosophical frameworks are key concepts in social science research method- 
ology as they refer to the system of beliefs researchers possess and which, in- 
evitably, play a central role in governing how a study is undertaken. These 
frameworks are often referred to as paradigms, a term that was made popular 
by Kuhn (1962). However, this term can lead to confusion due to the multiple 
ways it has been used over the years (Morgan, 2007, p. 48). Given this confu- 
sion, whenever this study’s philosophical framework is referred to in this vol- 


ume, the term worldview, which is commonly used in social sciences and in 
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the mixed methods literature, will be used. Worldview has been used as a 
synonym for a paradigm (Creswell, 2014, 2015; Teddlie & Tashakkori, 2009) 
and although worldview can be considered a broader term than paradigm, as 
it refers to “all encompassing ways of experiencing and thinking about the 
world, including beliefs about morals, values, and aesthetics” (Morgan, 2007, 
p. 50), its usage is less open to interpretation than paradigm. 

This study was set in a real educational context with real actors involved 
and the aim of the study was to create a blended skills-based process syllabus. 
Since the syllabus was and still is based on negotiation, its contents and lan- 
guage focus inevitably change from year to year depending on the input from 
each group of learners. Each individual learner is therefore able to contribute 
to the design of the English-language course so that it can become as relevant 
and useful for those learners as possible within the 30 hours that have been 
programmed for the course by the Faculty. Because of this real-world, practi- 
cal focus, the worldview that informed the conceptual framework of the study 
was pragmatism. Pragmatism has been described as “the search for practical 
answers to questions that intrigue the researcher” (Teddlie & Tashakkori, 
2009, p. 86) and has often been used in mixed methods research as its under- 
lying framework because pragmatism focuses on what works in practice. 
Pragmatism tends to concern itself with the “conceivable practical conse- 
quences” (Peirce, 1905/1984, p. 494) of the research, which differs from many 
other worldviews such as positivism or constructivism that “insist upon ante- 
cedent phenomena” in “reporting and registering past experiences” (Dewey, 
1931, p. 32 quoted in Cherryholmes, 1992, p. 13). In other words, in order to 
approach a context or study, a focus on practical consequences is favoured 
over a concern with preconceived ideas. The focus on practical consequences 
means that pragmatic research is characterised by choices made by the re- 
searcher in order to create those consequences, which means the specific hu- 
man values and needs that characterise the context are also considered and in 
fact precede the search for theories or descriptions. These choices often relate 
to how and what to research and what to do, but there are many ways to ap- 
proach a research question, many of which might not be suitable for those 
involved in the research or for the context itself. Therefore a pragmatist 


“simply eliminates them as possibilities for his or her classroom” because “not 
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everything that works is desirable, not every belief that is “true” is to be acted 
upon” (Cherryholmes, 1992, p. 13). In other words, pragmatists focus on what 
works in the context and the choices they make aim to be fit for the purpose(s) 
at hand. 

Pragmatism was also a suitable worldview for this study as after having 
worked so many years as an English-language teacher, from an ontological 
perspective I have learnt to adopt a pragmatic approach in the ESAP class- 
room. Although I can use many approaches to teaching, I have to focus on 
what works in practice due to the constraints of the context Iam working in, 
particularly with regard to the curriculum demands imposed on me by the 
university system I am working in, the number of students I have and the time 
allotted to the courses I teach. As far as the curriculum demands are con- 
cerned, and with regard to the study in question, the ESAP course I was and 
still am teaching must have a final examination that comprises a written and 
an oral component. This implies that the course should incorporate a focus on 
improving the specific speaking and writing skills and the specific language 
aspects that are to be assessed in the final exam. As far as the students are 
concerned, they are allocated into relatively large groups (on average between 
30 and 70 students in one class), which means that speaking activities that are 
designed to encourage most or all of the students to participate tend to have 
to be paired activities or small group activities rather than large group or 
whole-class speaking activities, thus optimising each individual student's op- 
portunities for language practice. Further, the course lasts 30 hours in total, 
and coupled with the varied language proficiency levels of the students, these 
also provide clear limitations as to what and how much language content is 
included in the course. Consequently, a pragmatic approach to my teaching 
context is the one that works best in practice. 


4.2 Approach to Research — Mixed Methods 


Pragmatism, the worldview governing this study, is described by Creswell as 
an “underlying philosophy that informs both quantitative and qualitative 
data collection” (2015, p. 16). Because this study was set in a real educational 
context and concerned the design of a negotiated syllabus for my advanced 
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English course for undergraduate Economics students, the conceptual frame- 
work of this study involved gathering data from different sources and at dif- 
ferent times. Therefore initial data were obtained from both qualitative re- 
search and quantitative research. The main data sources for the needs assess- 
ment and analysis stages of the study had to be first and foremost the students 
at the Faculty of Economics, of whom on average there were 600 per year over 
the three years this study covers. Needs assessment requires information not 
only about learners but also from learners and indeed Carkin states clearly that 
“needs assessment of the diverse learners in EAP underlies syllabus design” 
(2005, p. 87). In order to obtain as representative a depiction of the English- 
language needs of the population under study, i.e. the students at the Faculty, 
a sample from that population would have to be selected. As far as how large 
the sample had to be, Dérnyei states that “the often quoted ‘the larger the bet- 
ter’ principle is singularly unhelpful” (2007, p. 99) although Creswell disa- 
grees and emphasises that “with a large sample there is less room for error in 
how well the sample reflects the characteristics of the population” (2015, p. 
76). My aim was to obtain as many responses as possible in order to provide 
as representative a view as possible from the students that were following the 
degree programmes at the time and to draw tentative conclusions from the 
findings for current and subsequent students. Consequently, I decided to use 
a quantitative approach to collecting data from the students as using a quali- 
tative approach would have been impractical given the large number of stu- 
dents involved. 

However, the study’s aim was to gain as much of an insight as possible 
into the English being used at the Faculty and any problems students may 
have been encountering while studying in English, especially given the use of 
English as a medium of instruction to students who do not have English as 
their L1 and by lecturers who do not have English as their L1. Therefore, from 
a pragmatic and an ontological perspective, just approaching the students to 
provide data would have excluded an important target population: those lec- 
turers who use English as a Medium of Instruction (EMI). This group of peo- 
ple that interacts in and uses English with the students would be able to pro- 
vide their own in-depth, individual perspectives concerning the use of the 
English in their respective classrooms and so their voices were essential. This 
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target population only numbered 10 individuals in the year that the inter- 
views took place: they were all teaching completely different subjects from 
Accounting to Information Technology and so would have completely differ- 
ent experiences of how English was being used by their students. Moreover 
they all had differing academic, professional and even linguistic backgrounds 
and although some had been teaching in English at the Faculty for more than 
ten years, others had only been using EMI for a year or two. Thus it became 
clear that a more “person-centred enterprise” (K. Richards, 2003, p. 9), where 
these individual experiences and stories could be explored in depth through 
a more direct personal approach, would be beneficial. Therefore a qualitative 
approach, which is “directed at describing the aspects that make up an idio- 
syncratic experience” (Dérnyei, 2007, p. 126), was deemed more useful to col- 
lect data from the EMI lecturers, as this would provide “thick descriptions” 
(Geertz, 1973) and would allow me to understand more fully the contexts of 
each individual’s classroom and the resulting characteristics. 

The students and EMI lecturers were identified as the only target popu- 
lations from whom to gather data for this study as there were no other indi- 
viduals who were involved with the use of English for academic or teaching 
purposes at the Faculty, apart from me as the only ESP instructor. 

Given the specific nature of the context, therefore, the approach used in 
this study was a mixed methods approach. Put simply, mixed methods re- 
search (MMR) “involves different combinations of qualitative and quantita- 
tive research either at the data collection or at the analysis levels” (Dérnyei, 
2007, p. 24) while Johnson et al. go further and state that MMR “combines 
elements of qualitative and quantitative research approaches... for the broad 
purposes of breadth and depth of understanding and corroboration” (2007, p. 
123). By combining qualitative and quantitative research approaches, re- 
searchers endeavour to obtain a fuller understanding of the context under 
study, which has also been noted by Bryman as one of the main rationales for 
integrating qualitative and quantitative research (2006, p. 106). This desire to 
have as full an understanding of my context as possible is the overriding rea- 


son why I used a mixed methods approach. 
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Further it has been suggested that pragmatically-inclined researchers use their 
initial research questions as a way to select the research approach and meth- 
ods that would be most suitable, “enabling the investigation of important 
questions through mixing methods in ways that cannot be adequately ad- 
dressed with a single approach” (Riazi & Candlin, 2014, p. 142). Others 
(Tashokkori & Teddlie, 1998; Johnson & Onwuegbuzie, 2004; Teddlie & 
Tashakkori, 2009) also concur that pragmatism is “the underlying and the in- 
forming paradigm for MMR” (Riazi & Candlin, 2014, p. 142). 

Although a mixed methods approach combines features from quantita- 
tive and qualitative research, the mere combining of data from the two ap- 
proaches is not enough to label a study mixed methods. Mixed methods is 
used in an attempt to understand the subject under study more deeply and 
Creswell (2015, p. 2) states clearly that a mixed methods approach integrates 
aspects of the quantitative and qualitative approaches “and then draws inter- 
pretations based on the combined strengths of both sets of data to understand 
research problems”. This combined strength therefore provides a greater un- 
derstanding of the context under study than from just one or other approach. 
Given the complexity of my study: the different target populations, the indi- 
vidual perspectives within the samples taken from those populations, the 
number of samples, the different subjects taught in English as well as the 
lengthy timespan of the data collection, amongst other aspects, from a meth- 
odological perspective a mixed methods approach was thus deemed the most 
appropriate for this study. 


4.3 Research Design — Convergent Parallel Mixed Methods 


Within the mixed methods approach to research, Creswell (2015, p. 6) identi- 
fies three basic mixed methods designs: explanatory sequential design, where 
qualitative methods are used to help explain data from previously used quan- 
titative methods, and exploratory sequential design, where qualitative meth- 
ods are used initially to explore the context, perhaps because the questions to 
be asked are not known, and then these qualitative findings are used as a basis 
to create a second quantitative phase of the study. The third mixed method 
design is convergent design. In this type of mixed methods design both quan- 


titative and qualitative data are collected at the same time, analysed separately 
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and then merged “with the purpose of comparing the results (some say vali- 
dating one set of results with the other)” (Creswell, 2015, p. 6). Comparing 
the findings will then provide the researcher with a broader view of the topic 
and will indicate how far they confirm each other. Often convergent mixed 
methods design is also known as convergent parallel mixed methods design 
(Creswell, 2014, p. 219), to emphasise that the collection of data occurs at the 
same time as opposed to sequentially as in the other two mixed methods de- 


signs. This design can be illustrated thus: 


Interpretation 
QUANTITATIVE 
+ 


QUALITATIVE 


Compare 
and contrast 


Fig. 4.1 — Convergent parallel mixed methods design 


For this study, in the initial data collection stage, which corresponded with 
the first AR cycle (see Section 3.8.1 for details), the questionnaire to students 
and the semi-structured interviews with the lecturers who use English as a 
medium of instruction were conducted in parallel in the same period. How- 
ever, there were two further data collection stages, when the same question- 
naire was administered to the students. These stages occurred at the same time 
of year in the second and third year of the study in order to obtain a longitu- 
dinal view of students’ views. In order to avoid confusion when referring to 
the three different groups of students and thus the corresponding question- 
naire, they will be referred to as target population A, B and C (TPA/B/C) from 
now on in this volume (see also Figure 3.2 in Section 3.8). Consequently, alt- 
hough the first data collection stage can be considered a convergent parallel 
mixed methods design, the data collection stages of the project concerning 
TPB and TPC can be viewed as multiphase mixed methods design. This is of- 
ten used “in the evaluation or program implementation fields in which mul- 
tiple phases of the project stretch over time” (Creswell, 2014, p. 228), which 
was certainly the case in this study. Further data collection stages were intro- 
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duced throughout the three years to evaluate the implementation of the as- 
pects of the syllabus, and this will be discussed in more detail in the data col- 
lection section in this chapter. 

In this study, therefore, I have predominantly used the convergent paral- 
lel mixed methods design as a methodological principle. As I aimed to achieve 
a thorough analysis of not only what students have to study in English, but 
also the skills and language proficiency needed to study successfully in Eng- 
lish, and as we saw above, the respondents were both the students and their 
EMI lecturers, this convergent design, which occurred mainly in parallel, al- 
lowed for the merging of the data to achieve a certain degree of triangulation. 
Triangulation has been defined as 


intentionally using more than one method of data collection and analysis when 
studying a social phenomenon so as to seek convergence and corroboration be- 
tween the results obtained from different methods, thereby eliminating the bias in- 


herent in the use of a single method. (Riazi & Candlin, 2014, p. 144) 


The use of triangulation thus aims to lead to a fuller understanding of the 
subject under investigation, particularly when there is a variety of data 
sources and data collection methods, as was the case in this study. These 
sources and methods are explained further in Section 4.7. 


4.4 Research Tradition — Action Research 


Although the approach I used for this study is mixed methods, this approach 
was predominantly used for the initial and subsequent needs assessment and 
needs analysis stages. The chosen research tradition or design for this study, 
however, due to its aim to improve the existing course syllabus by adapting it 
through negotiation with the students, was action research. The two terms, 
tradition and design, are used throughout the literature to refer to similar con- 
cepts. K. Richards (2003, p. 13) lists seven core traditions in qualitative re- 
search: ethnography, grounded theory, phenomenology, case study, life his- 
tory, conversation analysis and action research. Creswell (2014, p. 13) also 
identifies grounded theory and ethnography in qualitative research but refers 


to them as research designs. In order to avoid confusion, in this volume I have 
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used “design” whenever I refer to the convergent design of the mixed meth- 
ods approach I used, and “tradition” to refer to action research. 


4.5 Approach to Data Collection 


The research approach for this study was mixed methods and the research 
design was convergent parallel mixed methods in the initial needs assessment 
phase in the first AR cycle. Therefore, the conceptual framework of this study 
involved data obtained from both qualitative and quantitative research. In or- 
der to conduct this initial needs assessment of the students at the Faculty, data 
were obtained qualitatively from the lecturers who use EMI in semi-struc- 
tured interviews and quantitative data were obtained from student target 
population A from an online questionnaire using the Opinio questionnaire 
software. The questionnaire was administered three times over a total of three 
years, corresponding to the three AR cycles as described in the previous chap- 
ter, so quantitative data were collected from three different student target 
populations (TPA, TPB and TPC). 

Apart from the data from the questionnaires, other data were collected in 
the second and third AR cycles from the students as a means to evaluate the 
implementation of aspects of the syllabus. Further discussion about these data 
collection methods can be found in Section 4.7 The Data Collection. 


4.5.1 Quantitative data collection - Sampling 


The convergent parallel mixed methods design, as discussed in Section 4.3, 
involves the collection of quantitative and qualitative data in parallel and is 
then merged in order to achieve triangulation. Often in this design “the quan- 
titative sample proceeds from a random or non-random sampling procedure, 
while the qualitative sample proceeds from purposeful sampling” (Creswell, 
2015, p. 78), which was reflected to a degree in this study. The quantitative 
data were collected from a questionnaire, a link to which was sent by email to 
a target population that comprised all undergraduate Economics students en- 
rolled in the two E&M and PPE degree programmes. These students were cho- 
sen because they were the students who had the ESAP course which is the 
focus of this study in their study plan as part of their undergraduate degrees. 
The ESAP course takes place annually in the first semester (October-January) 
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for the PPE students and in the second semester (February-June) for the E&M 
students during their second year of the degree programme. Although the tar- 
get population was the students who had the ESAP course in their study plan, 
the sampling was random as the questionnaire was to be filled in voluntarily. 

Consequently, the methodology used to collect data from the students 
was combining random sampling with a rational means of selection, in other 
words the deliberately chosen target population. This is stated as being “a 
particularly effective method for surveys with a specific focus” (Dérnyei, 
2003a, p. 73). 

However, when sampling is based on self-selection, which was the case 
in this study as the students who completed the questionnaires were all vol- 
unteers, problems can arise (Dérnyei, 2003a; Lowie & Seton, 2013). If a ques- 
tionnaire is completed based on self-selection, “the sample may lose its repre- 
sentative character” (Dörnyei, 2003a, p. 75) as the respondents who do com- 
plete the questionnaire will not necessarily be representative of the entire tar- 
get population. In the case of this study, it is possible that students with less 
motivation or those who had a lower language proficiency might have chosen 
not to undertake the questionnaire. Indeed Brown (2001, p. 85) points out that 
often the respondents who complete a questionnaire voluntarily may be gen- 
eralised as “eager-beaver” or “gung-ho” respondents and as such do not rep- 
resent the target population but just the “eager-beaver” or “gung-ho” part of 
that population. As can be seen from Table 4.1, which summarises the ques- 
tionnaires’ response rates, there was in fact a degree of dropout, where some 
students did access the questionnaire but abandoned it before completing it. 
Dörnyei (2003a, p. 76) suggests that to achieve sample representativeness in a 
survey that is based on self-selection, a response rate of 80% would be needed 
to guarantee a lack of bias. However, as the target population for this ques- 
tionnaire was limited to the Economics undergraduates and the findings are 
not being used to generalise beyond the specific context of this study, these 
samples can be seen as generally representative (Lowie & Seton, 2013, p. 53). 
Moreover, in Table 4.1, the response rate refers to the number of completed 
questionnaires as a percentage of the number of questionnaires accessed ra- 


ther than the number of emails sent. When the emails were sent to the students 
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inviting them to complete the questionnaires, the individual addresses re- 
trieved were not able to be identified as they were within group blocks di- 
vided by degree programme and year. As such, it was not possible for me to 
identify either the number of total emails that were sent, nor to whom they 
were sent and in fact it is likely that many of the email addresses were no 
longer valid or many of the emails were never even read by the addressee. 
Thus the reasons for the response rate being calculated based on the number 
of questionnaires accessed rather than the number of emails sent. Further as- 
pects concerning the questionnaires will be analysed in the discussion con- 


cerning the findings from the questionnaire data in Chapter 6. 


Table 4.1 — Questionnaire responses 


AR Cycle 1 AR Cycle 2 AR Cycle 3 

Timeline 6 - 31 May 2014 27 May - 15 June 2015 9 May - 9 June 2016 
Data QUANTITATIVE: QUANTITATIVE: QUANTITATIVE: 
collection | Online questionnaire Online questionnaire Online questionnaire 
method to target population A | to target population B | to target population C 
N9 218 141 147 
accessed 
Dp 151 99 115 
completed 
aia 69.2% 70.2% 78.2% 
rate % 

4.5.2 Qualitative data collection — Sampling 


Although convergent parallel mixed methods design is characterised by its 
use of qualitative and quantitative data collection methods, typically the same 
participants for this data collection are used (Creswell, 2014, p. 222; Creswell, 
2015, p. 78). However, due to the pragmatic underpinnings of this study, there 
were two distinct target populations chosen for data collection, the students 
and the EMI lecturers. Indeed, using different participants “works well when 
the intent of the convergent design is to compare different perspectives” 
(Creswell, 2015, p. 78), which was the intention of this study. Because these 
two target populations were entirely different from the perspectives of 


quantity, accessibility and ability to provide information, a quantitative 
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approach to data collection was used with the students, using a random 
sampling method, although the target population was very clearly defined. 
However, a qualitative approach was used to obtain data from the sec- 
ond, and much smaller, target population, the EMI lecturers. Creswell (2015, 
pp. 78-79) presents various methods for qualitative sampling in convergent 
parallel mixed methods design, including having both the quantitative and 
qualitative target populations of the same size, with the inherent problems 
that would cause in terms of resources and time for large qualitative samples 
and lack of representativeness for smaller quantitative samples. In the case of 
this study, however, the sampling for the qualitative data collection was rela- 
tively straightforward: the target population was clearly defined as it com- 
prised the members of staff who used English as a medium of instruction. 
Given the teaching model of the Faculty, where the three official teaching lan- 
guages of the University — German, Italian and English — are used approxi- 
mately equally throughout the degree programme, all the other lecturers 
would not have been able to provide any perspective on the English-language 
usage in their classrooms since they were teaching in either Italian or German. 
Therefore purposeful sampling was used to select the sample for the qualita- 
tive data, which amounted to ten participants. Below is a diagram to summa- 


rise the approach to data collection used in this study: 


QUANTITATIVE DATA QUALITATIVE DATA 
Questionnaire Semi-structured 


to students interviews 
random sampling with EMI lecturers 
Merged data 


Fig. 4.2 — Sampling used in this convergent parallel mixed methods design 


The aim of approaching the EMI lecturers was to gain a deeper insight into 
the English-language use and problems in the individual classrooms, and as 
the purposeful sampling only identified ten participants, conducting inter- 
views with all of them seemed to be the most appropriate method. One-to-one 
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interviews would help to “establish a relationship with people that enables us 
to share in their perception of the world” (K. Richards, 2003, p. 50). Further, 
the interview has been described as a “professional conversation” (Kvale, 
1996, p. 5) with the clear purpose “to obtain descriptions of the life world of 
the interviewee with respect to interpreting the meaning of the described phe- 
nomena” (Kvale, 1996, pp. 5-6). Consequently, selecting the EMI lecturers as 
participants in this study would allow for their perspectives to be shared, 
given that they are extremely important participants in the language-learning 
processes of the students in question. 

A further consideration regarding the gathering of the qualitative data, 
and to a certain extent also the quantitative data, was the issue of my role as 
the researcher in this study, given my multiple roles as teacher, colleague and 
researcher. How these roles were considered when creating a framework for 
the data collection and during the data collection itself will now be discussed 


in the next section. 


4.6 Researcher Roles 


A qualitative approach to research “is fundamentally interpretative, which 
means that the research outcome is ultimately the product of the researcher’s 
subjective interpretation of the data” (Dérnyei, 2007, p. 38). This means that 
the researcher should reflect on their role in the study and on how the data 
are collected and interpreted. Being reflexive means exercising self-awareness 
and “reminds the qualitative inquirer to be attentive to and conscious of the 
cultural, political, social, linguistic and ideological origins of one’s own 
perspective and voice as well as the perspective and voices of those one 
interviews” (Patton, 2002, p. 65). Given my role as the researcher for this study 
but also the teacher of the students (either concurrently or in the future) and 
the colleague of the lecturers who used English as a medium of instruction, I 
was fully aware of my multiple roles and potential problems that being an 
insider could generate. Therefore, as far as the quantitative data collected from 
the students were concerned, the questionnaire administered to the students 
was an online survey that guaranteed anonymity and students were under no 
obligation to complete it. This anonymity sought to address some of the 
disadvantages of questionnaires that have been highlighted by Dérnyei 
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(2003a, p. 13) including “acquiescence bias”, where respondents agree with 
statements that they actually might not really agree with and the “halo effect”, 
which is a tendency to overgeneralise. An example of the latter is where stu- 
dents who have a positive opinion of a teacher will state that everything about 
that teacher and class is positive, even if perhaps that is not the case. Certainly 
the opposite can also be true: if students do not like a teacher, they will prob- 
ably give a generally negative assessment of that teacher and his/her class. He 
also refers to “self-deception”, where people tend to deceive themselves as to 
their strengths and weaknesses (Dornyei, 2003a, p. 13). Given the fact that the 
questionnaire used in this study sought data about students’ proficiency lev- 
els through self-reporting, this aspect might have had an effect on the re- 
sponses provided. A further weakness of questionnaires is “social desirability 
bias” where respondents “try to meet social expectations and over-report de- 
sirable attitudes and behaviours while underrepresenting those that are so- 
cially not respected” (Dérnyei, 2007, p. 54). Given that the students complet- 
ing the questionnaires had been, were at the time or were going to be students 
of mine, they may have provided information that was not wholly accurate 
knowing that I would be reading the responses. However, the sampling of the 
data to generalise the results and ensuring anonymity were ways to address 
these issues. 

The follow-up surveys to students that are referred to further in this chap- 
ter were conducted on paper at the beginning of their written exam, but again 
students were under no obligation to complete them and they were also anon- 
ymous. It was impossible for me to identify individual responses due to the 
large numbers of students in the exam room (up to 90 in one exam). 

As far as the interviews with the lecturers were concerned, all the re- 
spondents were approached initially via email with an overview of the study 
and a request to meet for an interview. As one of the characteristics of an in- 
terview is for the researcher to create a relationship with the interviewee, in- 
terviews can be regarded as being “active” as they “are unavoidably meaning- 
making ventures” (Mann, 2011, p. 8). Garton & Copland (2010, p. 533) also 
focus on the fact that interviews “should be viewed as an interactional event 
in which the interviewer and the interviewee jointly construct meaning” but 


focus specifically on “acquaintance interviews” which are typically semi- 
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structured interviews “in which interviewer and interviewee have a prior re- 
lationship” (2010, p. 535). This prior relationship could be due to the research 
itself if it was a long-term project but Garton & Copland (2010, p. 536) state 
that acquaintance interviews can also comprise relationships between re- 
searcher and participant that include friends, colleagues and family members. 
In the case of this study, I had a variety of relationships with the interviewees: 
some were known to me purely because we were colleagues whereas others 
were colleagues and friends. Therefore, the typical asymmetry that there ex- 
ists between an interviewer and an interviewee, where the interviewer typi- 
cally asks the questions and the interviewee answers, is often not so prevalent 
in acquaintance interviews. Further, if the researcher knows the interviewee, 
“empathetic neutrality” (Patton, 2002, p. 49) could be compromised, which 
refers to the researcher’s stance as “middle ground between coming too in- 
volved, which can cloud judgment, and remaining too distant, which can re- 
duce understanding” (Patton, 2002, p. 50). Clearly knowing the interviewees 
could have led to my becoming too involved or jumping to conclusions from 


what the interviewees mentioned. 


4.7 The Data Collection 


As a mixed methods study with a convergent design, quantitative and quali- 
tative data were collected both in parallel and over time. Initial data were col- 
lected from two data sources: the students and the lecturers who use EMI for 
the needs assessment stage of the study, which have been used to inform the 
design of the syllabus. Further data collection stages occurred in the subse- 
quent AR cycles of the study in the evaluation stages as well as repeating the 
data collection from students for a further two academic years to provide both 
cross-sectional and longitudinal comparisons. 

As far as collecting data from the students was concerned, and as dis- 
cussed previously, I used a predominantly quantitative approach by sending 
an online questionnaire with closed questions to all enrolled economics stu- 
dents of that academic year (2013/2014). This questionnaire was used in the 
first AR cycle during the initial needs assessment stage, which coincided with 
the first year of the study and it was sent out at the same time of year (middle 
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of May for 4 weeks) for the subsequent two AR cycles in the following aca- 
demic years to gather data from three years’ worth of undergraduate students. 

Once the adapted syllabus was implemented in the second AR cycle of 
the study (2014/2015 academic year), evaluation was undertaken within the 
AR cycles to ascertain the students’ attitudes about the implementation of the 
new syllabus since there had been an extremely limited uptake. This was un- 
dertaken as a short end-of-course, paper-and-pen survey administered to the 
students at the beginning of their written exam at the end of the course. The 
feedback from this survey was analysed and further action was taken in AR 
Cycle 3 (2015/2016 academic year) of the study to address the points raised. A 
further, more detailed survey to the one in AR Cycle 2 was distributed to the 
students at the end of the PPE and E&M courses of the third year to evaluate 
their perceptions of the changes implemented and this one used a variety of 
open and closed questions a space for suggestions and comments at the end 
was also provided. This time, a mainly qualitative approach was employed 
through the use of the open questions in this course evaluation survey because 
this kind of approach “tell[s] the program’s story by capturing and communi- 
cating the participants’ stories” (Patton, 2002, p. 10, original italics). 

A further data collection stage occurred during the classes where negoti- 
ation of the syllabus actually took place. This comprised a worksheet that the 
students used during the negotiation class and which they submitted to me at 
the end of the class on a voluntary basis. The aim of this worksheet was man- 
ifold: it acted as a guide for the students’ interactions during the discussion 
phase of the negotiated class but for the purposes of the study and the sylla- 
bus, it provided evidence of what had been discussed and negotiated in class. 
Clarke states that “detailed written records would have to be kept during the 
creation and implementation of a Negotiated model” (1991, p. 22) and Hedge 
concurs, stating that when implementing a negotiated process syllabus, writ- 
ten records are essential for accountability issues: “when decisions about con- 
tent and procedure are taken throughout the course ... record-keeping be- 
comes crucial” (2000, p. 366). Further and most simply, if I had missed any of 
the input from the students, these worksheets would act as a reminder of what 
had been discussed. Moreover, given that the this study’s research design 
used a mixed methods approach to data collection, during these Negotiated 
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Portfolio classes, I observed the interactions in those classes and took field 


notes to document what occurred as “the act of writing itself imposes shape 
and substance” (K. Richards, 2003, p. 136) and, in the case of the notes taken 
in the class in October 2015, were then subsequently written up. These field 


notes were taken to provide as much documentation as possible about the in- 


teractions in the class and also the decisions taken together with the students 


during the negotiation phase. 


The Table 4.2 below summarises the data collection methods and data 


sources used. Further details about these data collection stages can be found 


in the following sections. 


Table 4.2 — Data collection methods and sources over the three AR cycles 


Data collection methods 


Data sources 


May QUAN: Online questionnaire (STP A) Students 
AR Cycle 1 | 2014 W WMI 
ay- QUAL: Semi-structured interviews 
August lecturers 
May QUAN: Online questionnaire (STP B) 
AR Cycle 2 | 2015 Students 
June QUAN: End-of-course survey 
QUAL: Worksheet in negotiated syllabus 
October | class 
QUAL: Field notes 
January | QUAN + QUAL: End-of-course survey 
2015 - ' ’ 
AR Cycle 3 2016 QUAL: Worksheet in negotiated syllabus Students 
April class 
QUAL: Field notes 
May QUAN: Online questionnaire (STP C) 
July QUAN + QUAL: End-of-course survey 
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4.7.1 Qualitative data collection 


For the initial needs assessment phase, and as far as the collection of qualita- 
tive data was concerned, the primary means of data collection in action re- 
search comprise the use of journals, interviews, documents and recording (K. 
Richards, 2003, p. 13). For this study, I used semi-structured interviews to 
collect data from the lecturers who use EMI that were audio-recorded and 
subsequently transcribed. I did not use journals as although they would have 
helped validate my results and be useful for further triangulation, I felt it 
would be impossible to persuade students or staff to keep a journal document- 
ing their experiences for this project. 


4.7.1.1 Semi-structured interviews 


Interviews are a typical means of data collection in qualitative approaches to 
research (D6rnyei, 2007, p. 24) and the qualitative interview is ever-present in 
applied linguistics (Mann, 2011, p. 6; K. Richards, 2003, p. 47). Given that 
mixed methods approaches to research combine both quantitative and quali- 
tative approaches to research, the interview is also a common means of data 
collection in mixed methods. Indeed Bryman noted as long ago as 2006 in his 
investigation of articles combining quantitative and qualitative approaches to 
research that “survey methods and qualitative interviews account for the vast 
majority of methods employed in the articles” (2006, p. 102) and he found that 
semi-structured interviews were the research method that was employed the 
most frequently. 

The semi-structured interview is one of various types of interviews used 
in applied linguistics qualitative research. The three main types of interviews 
where one interviewee is present are commonly referred to as unstructured or 
open interviews, semi-structured or semi-open interviews and structured in- 
terviews. An unstructured interview is often a relatively informal talk where 
the responses from the interviewee tend to dictate the direction and content 
of the interview. As such they are often used to gain deeper insights into the 
interviewees’ opinions, thoughts or experiences since an unstructured inter- 
view “allows maximum flexibility to follow the interviewee in unpredictable 
directions with only minimal interference from the research agenda” (D6- 


rnyei, 2007, p. 135). However, one of the main disadvantages of unstructured 
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interviews is that it is difficult to make useful comparisons between the an- 
swers provided by interviewees (Johnson & Weller, 2002, p. 499; K. Richards, 
2003, p. 184). A structured interview, on the other hand, whose “agenda is 
totally predetermined by the researcher, who works through a list of set ques- 
tions in a predetermined order” (Nunan, 1992b, p. 149), although much easier 
to compare across interviewees, tends to be very inflexible. Moreover, as K. 
Richards asserts, the qualitative interview ought “not merely to accumulate 
information but to deepen understanding and in order to do this the inter- 
viewer must be responsive to nuance and opportunity as the interview pro- 
gresses” (2003, p. 64). 

Therefore, a “compromise” (Dérnyei, 2007, p. 136) between these two 
types seems the most useful for this research and so semi-structured inter- 
views were used with the EMI lecturers as a means to administer a needs anal- 
ysis. The semi-structured format allowed for a great degree of flexibility, but 
still allowed me to ask the lecturers specific questions, the answers to which 
have been more easily comparable. Further, as all the interviewees are busy 
researchers and teaching staff, it was important to limit the amount of time 
being taken from their working day, especially as they certainly had little in- 
centive to provide time for this study as no immediate benefit would have 
been gained for them personally. Unstructured interviews can become very 
long whereas the semi-structured format allowed me to follow the research 
agenda, but provided sufficient flexibility for me “to follow up interesting de- 
velopments and to let the interviewee elaborate on certain issues” (Dérnyei, 
2007, p. 136). 

As these interviews were designed to be part of the broader analysis of 
students’ English-language needs, an interview guide was used. This kind of 
a guide is what K. Richards describes as “a resource that be drawn on in what- 
ever way and to whatever extent is appropriate” (2003, p. 69) and “helps make 
interviewing a number of different people more systematic and comprehen- 
sive by delimiting in advance the issues to be explored” (Patton 2002, p. 343). 
The interviewees were all domain experts but were not linguists so although 
they could provide information on the tasks undertaken in English in their 


classrooms and exams, they could not necessarily identify the linguistic re- 
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sources being used and would probably only be able to identify students’ lan- 
guage problems in a general way. Therefore, Hutchinson & Waters’s (1987, 
p. 59) target-situation analysis was adapted in order to function as a frame- 
work to collect information about the target needs of the students. Target 
needs, according to Hutchinson & Waters are “what the learner needs to do 
in the target situation”, which differ from learning needs which are “what the 
learner needs to do in order to learn” (p. 54). Consequently the “big questions” 
(K. Richards, 2003, p. 69) used in the interview guide were based predomi- 
nantly around the students’ target needs, which were designed to be more 
straightforward for the subject lecturers to articulate and describe since there 
was also the consideration that all of them do not have English as their first 
language. 

The interview guide was designed to follow a clear thematic structure to 
facilitate data collection and so the first section, entitled Your use of English 
sought to gather information about how each individual lecturer used English 
in the classroom given they all taught very different subjects with different 
year groups and class sizes. This question therefore aimed to elicit some of the 
students’ target needs in each course that used English as a medium of in- 
struction by analysing the descriptions of classroom interaction given by each 
lecturer. The interviewees therefore provided information concerning what 
teaching methodology they used, whether they used a traditional lecturing 
style or a more seminar-led approach, for example, as well as what teaching 
resources they used. In this section, the interview guide also provided the self- 
reflective question What problems do you have when you use English in your class? 
The phrasing of the question may appear insensitive since it assumes that the 
lecturers were having problems, but it was designed to encourage the lectur- 
ers to reflect on their classroom practice within the specific multilingual con- 
text in which they were working. The question aimed to identify whether any 
problems that did arise due to using EMI originated from difficulties the stu- 
dents experienced or from difficulties the lecturers themselves experienced as 
non-native English speakers lecturing to classes of non-native English speak- 
ers who may not even share their L1. Consequently although all the lecturers 
were non-native speakers of English, the question was not focusing on the fact 
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that they were non-native speakers, but more on the specific teaching context 
in which they were operating. 

The second section of the interview guide sought to focus more on the 
students’ use of English in each individual course and so the questions were 
designed to elicit further information about how English was used, how the 
subject’s exam was structured and most importantly the language problems 
that students encountered during classroom interaction, completing exercises 
and the exam. As such this part of the interview guide acted as a framework 
to conduct a present-situation analysis, which has the aim to “[estimate] 
strengths and weaknesses in language, skills, learning experiences" (Dudley- 
Evans & St. John, 1998, p. 125). The final section asked for reflection on the 
success of each course from a linguistic perspective and lecturers were asked 
to identify whether they would modify their courses in the future to take into 
account any problems they encountered with using English as a medium of 
instruction. This final part was designed to be a reflective element in the in- 
terview so that the lecturers could consider what actions could be taken in the 
future (either by them or even by me as the ESAP course tutor) to improve 
their course from a language perspective. 

Although as we have seen, an interview guide is often used to make the 
interviewing process of multiple respondents more systematic, the extent to 
how closely it is followed can vary from respondent to respondent. Often in 
semi-structured interviews “the interviewer remains free to build a conversa- 
tion within a particular subject area, to word questions spontaneously” (Pat- 
ton, 2002, p. 343), which is exactly what happened in this study. Therefore, 
although each interview was planned to last approximately 10 minutes and 
followed the same interview guide, they all had different running lengths and 
a variety of different information was provided, precisely because the answers 
that were provided then allowed me to probe further into the interviewee’s 
specific context. 

A further consideration regarding the interview guide was the fact that 
English is not the first language of any of the interviewees. Conducting inter- 
views in a participant's first language can be advantageous as it has been per- 


ceived to “lead to more open, easier, more expressive communication and to 
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elicit more or more accurate information” (Cortazzi et al., 2011, p. 510). There- 
fore, I had also designed the questions based on the target situation and the 
target needs of the students in the event that I would have to conduct the in- 
terview in Italian. Although my Italian is quite advanced conversationally, if 
I had had to conduct the interview in Italian, it would have been easier for me 
to ask concrete questions regarding when English is used, how it is used, what 
aspects are dealt with in English, how often it is used and so on rather than 
attempting to extrapolate more complex linguistic information, which I would 
not have been able to do effectively in Italian. In the end, though, all the inter- 
views were conducted in English despite all interviewees being given the op- 
tion of speaking in Italian before we commenced the interviews. The choice to 
use English arose because even though seven of the ten people interviewed 
have Italian as their first language (the others having French, Russian and 
Russian/Bielorussian), they all use English professionally both as a medium 
of instruction and in their other academic and research duties. Moreover, 
given that I had a prior relationship with all of them, they were all used to 
interacting with me in English and so it seemed natural to use English in this 
context. However, there can be problems when an interviewee is interacting 
in his/her second or third language, such as feeling less at ease or not neces- 
sarily having the language skills to express complex thoughts or recount ex- 
periences (Cortazzi et al., 2011, p. 509) and reference will be made to these 
issues when they arise in the Data Analysis chapter. 

The interview guide was also designed in this way so that this framework 
could facilitate the subsequent coding and categorising of the data, since the 
interviewees were all lecturers of completely different subjects. Asking situa- 
tion-based questions rather than more linguistic-based questions allowed me 
to obtain a more easily comparable picture of the tasks undertaken in English 
in the various classrooms and the problems the students face when carrying 
out these tasks. 

A final consideration about the semi-structured interviews was that as 
“there is almost universal agreement” (K. Richards, 2003, p. 67) that inter- 
views should be recorded, this is what occurred. When the EMI lecturers were 
approached initially, it was made clear to them that if they agreed to be inter- 
viewed, they would be audio recorded. Further discussion about the ethical 
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considerations of this aspect appears in Section 4.9. Therefore, each interview 
was recorded in its entirety on a digital recorder (iPhone) and the audio files 
were then saved for the following analysis stages. Another method of collect- 
ing data from interviews is to take notes, a method which is recommended 
especially if recording equipment malfunctions (Lankshear & Knobel, 2004; 
Patton, 2002). However, K. Richards states that taking notes during an inter- 
view is an “unnecessary formality ... which might inhibit the informant and 
distract the interviewer” (2003, p. 67) and it can be “a laborious task” (Lank- 
shear & Knobel, 2004, p. 200) particularly in longer interviews. Moreover, 
since the “raw data of interviews are the actual quotations spoken by inter- 
viewees” (Patton, 2002, p. 380), this would imply that the notes taken would 
have to accurately record what the interviewee says, which would have a neg- 
ative impact on the interactive nature of a semi-structured interview. As con- 
ducting interviews was a procedure with which I was relatively unfamiliar, 
and I would have the audio file afterwards to code and categorise and so could 
revisit the interviews, I therefore felt taking notes could be an unwelcome dis- 
traction. Consequently although I had the interview guide in front of me dur- 
ing the interviews, I did not take any notes at all. 


4.7.1.2 End-of-course surveys 


Another data collection method that was mainly qualitative, as summarised 
in Table 4.2, was the two end-of-course surveys, which were conducted dur- 
ing the written exams at the end of the courses in AR Cycle 2 and AR Cycle 3. 
The first survey was conducted due to the fact that only two students had 
opted to follow the newly designed syllabus that was used in AR Cycle 2, and 
all the other students chose to follow the existing syllabus. Therefore, it was 
crucial to understand why there had been such a low take-up of the new syl- 
labus in order to make adjustments for the following academic year as the aim 
was for the students to use the new syllabus. The survey consequently gave a 
list of possible reasons why the students had not chosen the modified syllabus 
and an “other” at the end for any reasons that had not been covered, and the 
students were requested to tick the reason that best covered why they had not 
chosen to follow the new syllabus. There was also a space for students to write 


a follow-up comment. 
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The second end-of-course survey, which was conducted at the end of the PPE 
course and the end of the E&M course at the end of AR Cycle 3, was more 
detailed in its design as it aimed to gather data in order to evaluate the modi- 
fied syllabus that was used in this AR cycle. This survey used a mixture of 
closed yes/no questions about the course and then asked follow-up open ques- 
tions to explain why the respondents had chosen yes or no. Due to the prag- 
matic underpinnings of this study, the purpose of using a more qualitative 
approach to data collection in this particular evaluation stage was to generate 
findings that would be useful for the subsequent course the following year. 
Patton states that in programme evaluation, “understanding the program’s 
and participants’ stories is useful to the extent that they illuminate the pro- 
cesses and outcomes of the program for those who must make decisions about 
the program” (2002, p. 10), and it is a qualitative approach to data collection 
that can better achieve this aim. 


4.7.1.3 In-class worksheet 


A further data collection method used in this study was the use of an in-class 
worksheet for the students during the Negotiated Portfolio classes with both 
groups of students in AR Cycle 3. As already mentioned, the worksheet had 
multiple aims; first it aimed to act as a framework for the students’ interac- 
tions during the discussion phase of the negotiated class because “for the 
learners to be able to plan their own learning they require not only the oppor- 
tunity to do it, but also information on how to design their plans” (Serrano- 
Sampedro, 2000, p. 113). For the purposes of the study and the syllabus itself, 
the worksheet was also designed to provide evidence of what had been dis- 
cussed and negotiated in class since in a negotiated syllabus, “record-keeping 
becomes crucial” (Hedge, 2000, p. 366). This worksheet was adapted from an 
example provided by Hedge, which was used in a writing course to modify 
the contents of a coursebook that was being considered for adoption in order 
to better meet the students’ needs (Hedge, 2000, p. 365). Although I was not 
using a coursebook for this course, I was using a book, Freakonomics by Levitt 
& Dubner (2005) so Hedge’s worksheet was adapted to focus on this book. 
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The questions on the worksheet were divided into three sections entitled 
“Yourself”, “Freakonomics” and “The Portfolio”, the latter being the name of 
the coursework that the students had to complete. The “Yourself” section re- 
quired the students to initially reflect on their strengths and weaknesses in 
English in an endeavor to raise their awareness of what individual language 
learning objectives they could address through the use of the Portfolio. In a 
learner-centred approach to syllabus design, learners need to be able to con- 
tribute actively to the planning of the syllabus but in order to do this, “the 
teacher has to encourage students to think critically about their learning expe- 
riences and about themselves as learners” (Tudor, 1993, p. 27). This initial re- 
flective phase that was integrated into the worksheet thus provided an oppor- 
tunity for the students to consider the purposes of the work to be done for the 
Portfolio, which is also the first element in Step 1 of Breen & Littlejohn’s ne- 
gotiation cycle (2000c, p. 32) as illustrated in Figure 2.1. As the negotiated syl- 
labuses aimed to address the needs of the individual students involved, re- 
flection and self-evaluation were considered crucial at this stage of the pro- 
cess. The students were requested to complete this section individually. 

The second section entitled “Freakonomics” was completed in small 
groups and students were required to discuss the contents of the book and 
select which chapters would be useful or interesting for them to study to- 
gether in class and which parts could be read without the support given in 
class. These two questions aimed to provide an initial focus for negotiation 
and for identifying the students’ needs, but also what Hutchinson & Waters 
refer to as their “wants”, i.e. what the learners feel they need or want from a 
course rather than the more objective necessities that are determined by the 
target situation (1987, pp. 55-56). This section also focused on aspects in the 
“contents” element of Step 1 of the negotiation cycle concerning the actual 
topics to be studied. Section three of the worksheet also focused on the content 
of the Portfolio, such as the type of texts to be written, as well as other ele- 
ments that comprise the negotiation cycle, particularly concerning the ways 
of working such as how many writing tasks there would be and the amount 
to be written. Consequently, although students were unable to choose the 
source material, Freakonomics, and the negotiated element of the course was 


the Portfolio, all other aspects were open for negotiation. 
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Once students had had time to discuss these questions together in groups, 
we came together as the whole class and we negotiated the elements to be 
included in the Portfolio. This also allowed for negotiation of content, proce- 
dures and evaluation means so that by the end of the class, we had decided 
how much writing would be undertaken, what type of writing would be un- 
dertaken and which chapters of the book would be read. 

At the end of the class, students were asked to submit their worksheets 
ona voluntary basis so that I could cross-reference what they had written with 
the notes I had taken during the negotiation stage and in total I received eleven 
worksheets. As mentioned previously, these worksheets were collected so that 
I had a record of what had been discussed and decided. During this class, I 
had also taken field notes while the students were working on the worksheet 
so that I had a further data collection method to cross-reference with the work- 
sheets completed by the students and the notes I had taken based on the ne- 
gotiation process. 


4.7.1.4 Field notes 


Field notes are a method of data collection in qualitative research which are 
“mainly written in the heat of the moment as events unfold before the re- 
searcher’s eyes” (Lankshear & Knobel, 2004, p. 229) and are the main data col- 
lection tool when observation is being undertaken. Field notes can either be 
taken at the time to be consulted during the analysis stage afterwards or they 
can be written up on detail after the observation period (Lankshear & Knobel, 
2004, p. 229), which can aid in the analysis stage as typed up field notes tend 
to be easier to work with subsequently. For this study, field notes were taken 
twice, once in the Negotiated Portfolio class in AR Cycle 2 and again in the 
Negotiated Portfolio class in AR Cycle 3. The field notes in AR Cycle 2 were 
typed up after the class but the second set of field notes were not typed up 
afterwards. 

Field notes function as a means to describe what is being observed and as 
such should avoid any judgmental language. They should describe the behav- 
iour being observed rather than falling “into the bad habit of primarily record- 
ing interpretations” (Patton, 2002, p. 303). This is because field notes should 


act as evidence of what was observed. In order to avoid assigning meaning to 
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what is observed, verbs should be favoured over adjectives when writing field 
notes (Lankshear & Knobel, 2004, p. 231; Patton, 2002, p. 303) so that the spe- 
cific actions observed are recorded rather than the researcher’s interpretation 
of those actions. Direct quotations of what is said are also recommended as 
they provide an “emic perspective” (Patton, 2002, p. 303) — an insider’s per- 
spective — to what was being observed. Field notes can also contain the ob- 
server's own feelings and opinions since in qualitative research “the ob- 
server’s own experiences are part of the data” (Patton, 2002, p. 304). K. Rich- 
ards (2003, p. 137) also recommends that analytical and relational aspects are 
also recorded. Analytical aspects would relate to insights and connections 
with theory whereas relational issues concern how the researcher connects 
with what is being observed and the participants being observed, as well as 
any personal reflections on the observation. Given that I was observing my 
own students in this case, the field notes I took contained much of what is 
recommended. Analysis of these field notes can be found in Section 5.6.2.2. 


4.7.2 Quantitative data collection 


There are different means to collect quantitative data in applied linguistics, 
including language tests and objective measurements of a specific phenome- 
non, but due to the nature of this study, another very common method was 
used: the questionnaire. In the literature, there is an array of terms often used 
interchangeably for this means of data collection, including questionnaire, 
questionnaire survey or survey (Dérnyei, 2003a, p. 5), but for this volume I 
shall use the term “questionnaire”. 

Questionnaires can take on many guises: they can be seen as interview 
plans where the questions are read out to the interviewee, which would typi- 
cally take place in a structured interview, or they can be “self-administered 
pencil-and-paper questionnaires” (Dörnyei, 2003a, p. 6) although nowadays, 
with the almost universal use of email and the internet, many tend to be ad- 
ministered online. Moreover, depending on the type of information sought, 
Lankshear & Knobel (2004, p. 164) identify purely descriptive questionnaires, 
cross-sectional questionnaires or longitudinal questionnaires. Descriptive 
questionnaires simply describe the current situation of the sample, a cross- 


sectional questionnaire takes a targeted sample and infers from the responses 
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the typical situation for the broader group or population, whereas a longitu- 
dinal questionnaire focuses on one specific group over time. For this study, a 
cross-sectional questionnaire was used which was sent to a target population 
that comprised all undergraduate Economics students enrolled for the degree 
programmes in Economics and Management and in Politics, Philosophy and 
Economics. As discussed, the questionnaire was then administered in the two 
following academic years, 2014/2015 and 2015/2016 to obtain a longitudinal 
view of the students’ English language proficiency and any problems they 
may be experiencing studying in English at the Faculty, with a view to also 
use the data as an evaluation tool for the adopted syllabus. 


4.7.2.1 Strengths and weaknesses of use 


Questionnaires are commonly used as there are many benefits to their use, 
although undoubtedly there are also many drawbacks. Advantages of using 
questionnaires are that they can be sent to a large number of respondents and 
are versatile, especially now that many can be administered online and can 
potentially reach an extremely large number of respondents thanks to email 
and social media. For this study, an online questionnaire was used in order to 
reach as many students as possible, and also because it allowed for completion 
whenever and wherever was convenient for the respondent. Using an online 
questionnaire also facilitated the processing of the collected data afterwards. 
However, one problem that can be experienced with online questionnaires is 
that response rates can often be very low (Dérnyei, 2007, p. 101; Brown, 2009, 
p. 214). Therefore, in order to mitigate this problem, several emails were sent 
as periodic reminders, which seemed to contribute to the relatively high re- 
sponse rate in the three years surveyed, which ranged from 69.2% to 78.2%, as 
was shown in Table 4.1. 

Another disadvantage of questionnaires is that often their reliability can- 
not be guaranteed, either due to ambiguous question wording, or to unreliable 
or inconsistent answers from respondents. In order to avoid ambiguous, 
loaded or simply unclearly worded questions because “in questionnaires so 
much depends on the actual wording of the items” (Dérnyei, 2007, p. 112), the 
questionnaire was piloted, a stage that is regarded as being fundamental 
(Nunan, 1992b, p. 145; Dérnyei, 2003a, p. 63). An initial piloting phase was 
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used involving eight Economics students, and therefore potential respond- 
ents, who had volunteered for the task after I had appealed for help during 
my classes. These students were given the pilot questionnaire on paper and 
were asked to give their responses as real respondents, but also to focus care- 
fully on the questions so that they could provide feedback on whether there 
were redundant or irrelevant questions. They were also asked to provide feed- 
back on the contents and order of the questions and the language used, given 
the fact that the questionnaire was administered in English. English was cho- 
sen as the questionnaire language because the students were all studying at 
the trilingual Faculty of Economics, which means they had all had to certify 
at least a B2 level in English in order to enrol for the undergraduate pro- 
gramme. Further, the cohorts involved in the study had a multitude of L1s, 
mainly German and Italian, but also Ladin, Spanish, Albanian and Russian 
amongst others, so the prospect of having to produce a version in all these 
other languages was too vast to contemplate given the scope of this research. 

Once the students had completed the draft questionnaire, we discussed 
its contents. Much of the feedback from the students concerned the questions 
that refer to what they have to do in English during their studies. I had pre- 
pared a list of tasks that I believed students had to complete for their courses 
but the students provided further details. In fact for the part of question 10 
which asked what do you have to do in English when at the unibz — writing, the 
task that scored the highest response rate in questionnaire 1 and 2 and the 
second highest in questionnaire 3 was write exam answers. This task was added 
due to the students’ suggestions in the piloting phase. 

Further disadvantages of questionnaires have been highlighted by Dö- 
rnyei (2003a, p. 13) concerning the behaviour of respondents especially if the 
researcher is known to them. These aspects, “acquiescence bias”, the “halo ef- 
fect”, “self-deception”, and “social desirability bias” were discussed in Section 
4.6 when reflecting on the multiple roles that I assumed in this study. Further 
reference to these points will occur in Chapter 6 when discussing the findings 
generated by the questionnaires. 
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4.7.2.2 Design and objectives 


As far as the design of questionnaires is concerned, they can contain open 
questions, closed questions, or a combination of both. As Nunan (1992b, 
p. 143) observes, “while responses to closed questions are easier to collate and 
analyse, one often obtains more useful information from open questions” and 
Long (2005c, p. 39) echoes this: “closed items provide standardized, easily 
coded and quantified data, but they may limit possible responses and may 
result in overly simple treatments of complex issues”. However, although 
open questions may provide extremely interesting and more accurate or rele- 
vant responses, respondents need more time to complete them. Additionally, 
for this study there were few incentives for the students to complete the ques- 
tionnaire because they themselves were unlikely to reap immediate or even 
future benefit from any findings or results from this work. Many who com- 
pleted the questionnaire had already attended or were attending the ESAP 
course and this aspect might have affected the response rate to a degree. 

Further, the choice to conduct the questionnaire in English meant that if 
there had been a focus on open questions, the respondents would probably 
have felt they should have answered in English. It is likely that if the ques- 
tionnaire had required respondents to answer open questions in English, 
some would have abandoned the questionnaire early on as they may have felt 
I, as their current (or future) English tutor, would judge them on the accuracy 
of the English used. Therefore closed questions were used in the questionnaire 
in order to encourage as many responses as possible, as well as to facilitate the 
collection and analysis of the data. 

The questionnaire aimed to identify the common features of the students’ 
English language use and any problems they might have in the language as 
well as provide some answers to the first research question. The closed ques- 
tions that were used to obtain the variables that would then be analysed cov- 
ered various categories. Classification questions, which are questions about 
the respondents themselves, included asking them to identify which degree 
programme they were following, which year of study they were in, whether 
they had any international English-language certificates, their age, sex and 
first language. These questions were included so that these independent vari- 
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ables could be analysed subsequently to identify whether they had any rela- 
tion to the responses to some of the other questions, which were attitudinal or 
behaviour-related questions about studying in English and English-language 
proficiency levels. These classification questions were also included so that 
they could be compared across the three questionnaires over the three years 
to establish how generalisable the findings could be for any adaptations to the 
syllabus during and after the study. 

Many of the other questions contained in the questionnaire were attitudinal 
questions, such as question 20 “How difficult do you find the following in 
English?” and behaviour-related questions mainly referring to the frequency 
of engaging in the various language skills in questions 7, 12, 14 and 19. These 
attitudinal questions were used as the questionnaires sought to explore the 
respondents’ opinions about their own English proficiency and the level of 
proficiency they felt was needed to study at the university. These questions 
were also designed to provide responses that could be compared with the re- 
sponses from the EMI lectures in an attempt to achieve triangulation between 
the data sources. The behaviour-related questions, however, were used to in- 
vestigate whether there was any relationship between the amount of time the 
respondents stated they were practising the skills needed to study in English 
and their self-reported level of difficulty in each skill. As such they aimed to 
explore the second research question, which was “What skills practice should 
be maintained or enhanced in the syllabus?” 

Other questions contained in the questionnaires aimed to provide further 
information about what effectively students had to do in English in the four 
skills (questions 8-11), where a list of activities were provided and respond- 
ents could choose as many as were pertinent. Again, these questions were in- 
cluded so that they could be compared with the responses provided by the 
EMI lecturers. Other questions also sought to investigate respondents’ moti- 
vation towards learning English (questions 30-33). 

In order to gather this information, the questions were presented in dif- 
ferent formats depending on whether the questions were classification, attitu- 
dinal or behaviour-related questions. Certain classification questions were 
simply presented as yes/no questions (e.g. question 4: Do you have an inter- 
national language certificate?) or as drop-down lists such as in the questions 
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seeking to identify the respondents’ L1, their age and their year and pro- 
gramme of study. Other lists were also used in questions 8-11 where respond- 
ents were asked to indicate what type of writing tasks they undertook, for 
example. Other questions aimed to obtain respondents’ attitudes about using 
English in their university studies, so various rating scales were used, which 
“require the respondent to make an evaluative judgement of the target by 
marking one of a series of categories organized into a scale” (Dérnyei, 2003a, 
p. 36). Rating scales are often used in applied linguistics as “they can be used 
for evaluating almost anything” (Dérnyei, 2003a, p. 36) and as one of the aims 
of the questionnaires was to obtain evaluative responses from the students, 
different types of rating scales were used for this study. Question 20 used a 4- 
response Likert-type scale to answer the question “How difficult do you find 
the following in English?”, referring to the four skills, reading, listening, writ- 
ing and speaking and providing four response options — “very difficult, quite 
difficult, quite easy, very easy”. Original Likert scales are used in question- 
naires to gauge respondents’ attitudes to statements by asking them to indi- 
cate how strongly they agree or disagree with the statement and there are usu- 
ally five response items: strongly agree — agree — neither agree or disagree — 
disagree — strongly disagree (Dörnyei, 2007, p. 105). As the aim of this partic- 
ular question was to obtain students’ attitudes about how difficult they found 
the individual skills, the scale was represented as a Likert scale, but used an 
even number of responses to avoid respondents taking the middle category 
that offers them the opportunity not to make a real choice. Research indicates 
that approximately 20% of respondents are likely to make this “undecided” 
choice, (Dérnyei, 2003a, p. 38), and as I felt the response “neither difficult nor 
easy” would not be providing any useful data for this study, four responses 
were included. Moreover, although this rating scale for question 20 was rep- 
resented as a Likert-type scale, it was more similar to a semantic differential 
scale, which is a “continuum ... between two bipolar adjectives at the ex- 
tremes” (D6rnyei, 2007, p. 105). Generally they are represented as can be seen 
below, where the respondents then place their mark or X in the corresponding 
position on the scale, but the questionnaire software I used did not allow me 


nou 


this option and therefore I chose to scale them as “very difficult”, “quite diffi- 
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cult”, “quite easy”, “very easy”. 
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In order to obtain respondents’ perceptions of what their proficiency was in 
the four skills in English and what they felt the corresponding levels needed 
to study at the university were, in questions 21-24, a different rating scale was 
used: “Level of competence in English. Please indicate your level (from + to 
++++++)”. These six levels were based on the levels of the Common European 
Framework of Reference (CEFR) so that + corresponded to A1, ++ to A2 and 
so on, and were articulated in a similar way to the “can do” statements from 
the CEFR’s self-assessment grid (Council of Europe, 2001, pp. 26-27). How- 
ever, they were rewritten to focus more specifically on what the students “can 
do” in English for their studies so, for example, the descriptor in the question- 
naire for “Speaking ++++ “(which corresponded with the B2 level from the 
CEFR) read “I can take an active part in discussions in familiar contexts and 
express myself clearly on subjects related to my studies and interests”. This 
adaptation of the original descriptor integrated and foregrounded the element 
of studies to make it more specific to the aims of the questionnaires, which 
were to obtain data on the students’ self-reported proficiency levels and the 
problems they may or may not have when studying in and using English at 
the university. 

Further considerations concerning the design of the questionnaire in- 
cluded ensuring that the general introduction explained clearly the purposes 
of the questionnaire and that the responses were confidential and anonymous. 
The layout was largely dictated by the software that was used but I did ensure 
that the questions were organised thematically and followed a logical se- 
quence: 

- general information about the student’s university education 

- international language exams 

- how often and for what they use English at the university 

- how often and for what they use English away from the university 

- how difficult they find the four skills 

- their perceived proficiency level in the skills and how this matches the 


required level 
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- problems they experience with English at the university and away from 
the university 

- what they are doing to improve their English 

- personal classification questions (sex — age — L1) 


The personal classification questions were placed at the end of the question- 
naire as from an ontological perspective, I concurred with Dérnyei (2007, 
p. 111) and felt these personal questions could be potentially off-putting for 
the respondents if placed at the beginning of the questionnaire and could be 
construed to partially contradict the promise of anonymity in the introduc- 
tion. 

As far as the layout is concerned, moreover, the items in the rating scales 
were varied both in where they were placed in the questionnaire (the rating 
scales almost always appeared singularly and were preceded and followed by 
lists, except for questions 21-24) and how they were presented. This was done 
“to create a sense of variety and to prevent respondents from simply repeating 
previous answers” (Dornyei, 2007, p. 111). When analysing the data, it be- 
comes clear that some of the respondents did seem to repeat previous an- 
swers, but discussion of that aspect can be found in Chapter 5. 


4.8 Approach to Data Analysis 


The data that were collected in this study occurred in various forms: qualita- 
tive data in the form of the coding and categorising of interview transcripts 
and end-of-course survey responses, and field notes from classroom observa- 
tion, and quantitative data mainly from the various questionnaires adminis- 
tered over the AR cycles that characterise this study. Therefore, I have used 
different approaches when analysing the data from each collection method. 

As this study used mixed methods as a research design, and specifically 
a convergent parallel design, the data collected in AR Cycle 1 were analysed 
“to address the mixed methods question” (Creswell & Plano Clark, 2007, 
p. 128), in other words the data from the two approaches were analysed sep- 
arately and then merged to identify any commonalities or discrepancies be- 
tween the data. 
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4.8.1 Approach to data analysis — Qualitative data 


4.8.1.1 Semi-structured interviews 


The main qualitative data collected for this study were from the ten EMI lec- 
turers in the form of semi-structured interviews in AR Cycle 1. According to 
K. Richards et al., data analysis in qualitative research “involves a process of 
breaking down and building up the available data” (2012, p. 77). If the data is 
written, as is the case in this study with the interview transcripts, the data has 
to be coded so that categories and concepts can be identified and any relation- 
ships that there are can then be extrapolated. In order to code the data, there 
are two differing methods: the deductive and the inductive method. The de- 
ductive method looks at the data from the categories already decided upon by 
the research questions, and any references to these categories are then identi- 
fied. The inductive method on the other hand, uses the data to generate the 
categories. Altrichter et al. suggest that both methods can be used in AR so 
that the researcher remains “open to the surprises the data can contain” (1993, 
p. 124) and Creswell & Plano Clark assert that “inductive and deductive think- 
ing” are often combined in mixed methods research design due to its practical 
nature (2007, p. 10). Predominantly for this study, a deductive approach was 
used to generate data as “often what researchers are doing is checking 
hunches” (Gibbs, 2007, p. 5) and being the instructor of the course, I was aware 
many students had problems both with studying in English and studying Eng- 
lish. It was for this reason that the interview guide used in the interviews 
asked the same questions to all respondents and concentrated mainly on in- 
vestigating the target situation and the language problems that arose in each 
interviewee’s classroom. However, given the semi-structured nature of the in- 
terviews and the unexpected directions that some interviews took, categories 
were also generated inductively during the subsequent analysis of the data. 
In order to analyse the spoken data collected in the interviews, categorical 
analysis and coding were used. Categorical analysis and coding can be used in- 
terchangeably, but Lankshear & Knobel define categorical analysis as “the sys- 
tematic organization of data into groupings that are alike, similar or homoge- 
nous” (2004, p. 270) and it “refers to the process of developing and applying 
codes to data” (2004, p. 270). Coding, on the other hand, refers to applying codes 
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to information that belongs to the different categories identified; they are “labels 
that assign symbolic meaning to the descriptive or inferential information 
compiled during a study” (Miles et al., 2014, p. 71). Categorical analysis is “an 
iterative process” whose aim is to “identify logical relationships among 
categories of items in order to refine the number of categories to be used in 
writing up the study” (Lankshear & Knobel, 2004, p. 271). Certainly in this 
study, the analysis of the spoken data followed an iterative process and will be 
discussed further in the data analysis chapter. 
As far as coding is concerned, there tend to be three types of coding, known as 
open (or initial) coding, axial coding and selective coding. Open coding is the 
stage where the data is broken down and codes are affixed to the data 
transcribed. Axial coding is the organisation stage of coding, where categories 
are related to other categories or sub-categories and any connections that appear 
can be made. The axis of the axial coding is therefore a category. Selective coding 
is the stage where “a central category... is identified in terms of which other 
categories can be refined and unintegrated” (K. Richards, 2003, pp. 276-277). 
The open coding, which tends to be descriptive rather than analytical, was 
completed once the interviews had been transcribed. The texts were inserted 
into the NVivo programme and codes were allotted following the common 
themes that were referred to by the interviewees based on their responses to 
the questions from the interview guide. In vivo and descriptive coding were 
used in the open coding phase. In vivo is when the exact word or phrase used 
by the interviewee serves as a code. In order to distinguish in vivo codes, they 
are normally placed between inverted commas, which is also the convention 
adopted in this volume. In vivo coding identifies words or phrases that seem 
to be conspicuous, particularly if they are used repeatedly by participants, as 
these “often point to regularities or patterns in the setting” (Miles et al., 2014, 
p. 74). An example of in vivo coding is in the following example from interview 
transcript 1, which describes a strategy the lecturer used to mitigate students’ 


comprehension problems in his class: 


Yeah so after the first time I understood that 1 “only way” 
the t only way to get them to understand 2 “first” 
something was to ? first show all the footage 


3 4 3 “then” 
and ° then talk about what was told and 4 then 
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show it again — > piece by piece discussing 4 “then” 
what we had just heard — 5 “piece by piece” 


Descriptive coding is when a label is assigned to a word or short phrase to 
summarise the main topic of the utterance. Often descriptive coding uses a 
noun or noun phrase as the code, which then “provide an inventory of topics 
for indexing and categorizing” (Miles et al., 2014, p. 74). An example of de- 
scriptive coding can be seen with the same extract as before: 


Yeah so after the first time I understood that ! 
the only way to get them to understand some- 
thing was to first show all the footage and 
then talk about what was told and then show 
it again — piece by piece discussing what we 
had just heard — 


1 strategy 


Once this stage had been completed, axial coding occurred where the main 
thematic categories across the spoken data were identified and analysed. This 
stage occurred manually through the addition of notes and highlighting tech- 
niques on the transcripts. Selective coding was then undertaken again through 
the application of labels and notes to the transcripts and the main categories 
were further analysed and identified. The various phases of coding and cate- 


gorising are illustrated in the data analysis framework in Section 5.2. 


4.8.1.2 The end-of-course surveys 


The end-of-course surveys were the other main dataset that were analysed 
predominantly qualitatively. To analyse the first end-of-course survey, basic 
descriptive statistics and coding were used due to the fact that the students 
just had a choice of 7 responses (plus an “other (please specify)”) and the aim 
of the data collection was to understand why students had not chosen to en- 
gage with the first version of the negotiated syllabus. The second end-of- 
course survey that was conducted with both the PPE and E&M students and 
then analysed at the end of AR Cycle 3 together were a mixture of closed and 
open questions, as discussed in Section 4.7.1 and so content analysis was used, 
which is “a research method that uses a set of procedures to make valid infer- 
ences from text” (Weber, 1985, p. 9). In this case, it was used inductively, in 
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order to “search through the text, word by word, phrase by phrase, or para- 
graph by paragraph to distil important ideas and themes therein” (Lankshear 
& Knobel, 2004, p. 336). The conceptual framework that evolved from the cod- 
ing of this second end-of-course survey is presented and discussed in Sections 
5.6.1 and 7.5.6. 


4.8.2 Approach to data analysis — Quantitative data 


The initial data obtained for the needs assessment were from both quantitative 
(questionnaire) and qualitative (semi-structured interviews) approaches and 
were collected in parallel in the first AR cycle. Following the convergent 
mixed methods design, they were then analysed and merged to provide a 
more complete analysis of the research problem. However, given the various 
phases of this action research study, the quantitative data were collected at 
different points throughout the research, as was illustrated previously in the 
Data Collection table (Table 4.2). The first questionnaire from student target 
population A in AR Cycle 1 generated 151 completed responses from 218 ac- 
cessed. The raw data were prepared by transferring them from the question- 
naire programme into an Excel file where they were organised and coded. Ex- 
cel allows the user to easily sort data, so the completed responses were sorted 
following the degree programme: first PPE and then E&M so that the two stu- 
dent cohorts would be listed together. The sequence of the responses within 
each degree programme was dictated by the respondent number that the ques- 
tionnaire software had assigned to each respondent chronologically when 
they logged into the questionnaire, and which I had no control over. Moreover 
the allocation of this respondent number ensured the anonymity of the partic- 
ipants. 

The partially completed questionnaires were not included in the analysis 
as they had all been abandoned at different stages of the questionnaire (and 
some had been accessed but not even the first question had been answered). 
Missing data can be a “nuisance” (Dornyei, 2007, p. 205) because it is often 
unclear whether the missing response is deliberate or has been an oversight 
on the part of the respondent. Moreover, when many variables are examined, 
which is the case of this study, there tends to be the necessity for every re- 


spondent to have values for each variable, otherwise these values are excluded 


86 


Research Methodology 


from the analysis (Dornyei, 2007, p. 205). Due to having a great amount of data 
with the completed responses, therefore, only these were analysed and all par- 
tially completed responses were discarded. 

Once quantitative data are sorted, it is important to compile a “coding 
frame that specifies the meaning of the scores for each item” (D6rnyei, 2003a, 
p. 98). A coding frame is a means to provide an answer with a numerical score 
so that subsequently statistical packages can analyse the results. With the 
transfer of the data from the questionnaire software to Excel, this phase was 
almost completely automatic. Consequently, for question 20, “How difficult 
do you find the following in English?”, referring to the four skills, reading, 
listening, writing and speaking, the four response options — “very difficult”, 
“quite difficult”, “quite easy”, “very easy” — were automatically transferred 
into 1, 2, 3, 4. The responses to other questions that used drop-down menus, 
such as the last question that asked for the L1 of the respondent, were trans- 
ferred into the Excel file as they had appeared in the questionnaire: German, 
Italian, Ladin, Germant+Italian, Other (with the language as entered by the re- 
spondent). 

However, it was clear that Excel would not be able to analyse effectively 
the data that I had collected and so the data to be analysed was then inserted 
into the Statistical Package for Social Science (SPSS) programme, version 24, 
which was deemed to be a more suitable tool for the type of data analysis 
needed. As this had to be done manually, a coding frame then had to be com- 
piled for the selected data to be analysed. Therefore for the question asking 
respondents if they had an international English-language certificate, for ex- 
ample, the answer was either yes or no and so yes was coded 0 and no was 
coded 1. For question 20, “How difficult do you find the following in Eng- 
lish?”, referring to the four skills, reading, listening, writing and speaking, the 
four response options — “very difficult”, “quite difficult”, “quite easy”, “very 
easy” — were allotted the same codes as in the Excel file, so 1, 2, 3, 4. Although 
it has been suggested that different variables should be labelled with a differ- 
ent coding frame (Dérnyei, 2003a), the codebook that was compiled for the 
SPSS data was clear and so there was no confusion. The codebook is a means 
to “provide a comprehensive and comprehensible description of the dataset” 
(Dérnyei, 2003a, p. 100) and can contain various pieces of information about 
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the variable including its name and coding frame. For this study the following 
codes were allotted to each response: 

- name of variable 

- coding frame 


- whether the variable was nominal or ordinal 


Once all the data had been inserted into SPSS and the codes had been allotted, 
statistical analyses were run to investigate the data and generate findings. 
Given that the questionnaire data had been collected to explore what language 
problems the students felt they had while studying in English at the Univer- 
sity, as well as to investigate any relationship between how much they stated 
they practised English and their self-reported levels, a series of descriptive 
statistics were generated to examine whether there were any relationships be- 
tween the variables. Further analyses were run to ascertain whether there 
were any relationships between the students’ L1, their degree programme and 
their self-reported difficulties they had studying in English. A detailed de- 
scription of the data analysis of the questionnaires can be found in Section 5.4. 

As there were two other questionnaires conducted over the course of this 
study, the whole process described above was repeated in AR Cycles 2 and 3 
over the subsequent two years. Once all the data had been inserted into SPSS, 
further statistical analyses were run to compare the data across the three ques- 
tionnaires, as well as to identify any patterns or deviances. Full details of these 
tests are provided in the Data Analysis chapter. 

A further point about the analysis of both the quantitative and qualitative 
data is that data reduction or “data condensation” (Miles et al., 2014, p. 12) 
was performed during this study. Data reduction “involves reducing the di- 
mensionality of the qualitative data ... and quantitative data” (Johnson & 
Onwuegbuzie, 2004, p. 22) through a process of selection, focusing and trans- 
formation. Miles et al. prefer the term “data condensation” because data re- 
duction “implies we’re weakening or losing something in the process” (2014, 
p. 12) and in fact data reduction makes the data stronger as it is a method of 
analysis that “sharpens, sorts, focuses, discards and organizes data in such a 


way that “final” conclusions can be drawn and verified” (2014, p. 12). Data 
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reduction progressed throughout the data analysis stages due to the various 
stages of coding and thus contributed to the iterative nature of the process. 


4.8.3 Approach to data analysis — Mixed methods 

Convergent parallel mixed methods design 
Given the convergent parallel mixed methods design for the research in the 
initial needs assessment stage, the quantitative and qualitative data were an- 
alysed separately, as described in the preceding two sections and then the re- 
sults were compared “to see if the findings confirm or disconfirm each other” 
(Creswell, 2014, p. 219) as illustrated in Figure 4.1 previously. 

Consequently in this design, once the quantitative and qualitative data 
have been collected, analysed and the results have been represented, the re- 
searcher has to merge the two datasets so that “a more comprehensive ac- 
count” (Bryman, 2006, p. 106) of the situation can be obtained. In this way the 
research questions that are associated with the mixed methods design can be 
answered more fully. When merging these datasets, it is important to investi- 
gate to what extent the quantitative and qualitative data merge and what sim- 
ilarities and differences there are between the data. Creswell & Plano Clark 
suggests that there are two ways to merge quantitative and qualitative data: 
either to transform one type of data into another so it can be easily merged 
and then compared across the datasets or to compare the data through a dis- 
cussion (2007, p. 137). The data transformation stage, where qualitative data 
is turned into numerical code that can then be represented statistically, or 
“quantitized” (Tashokkori & Teddlie, 1998), or where quantitative data is con- 
verted into narrative data that can be analysed qualitatively, or “qualitized” 
(Tashokkori & Teddlie, 1998), is regarded by some as optional (Johnson & 
Onwuegbuzie, 2004). Others (Riazi & Candlin, 2014; Creswell & Plano Clark, 
2007) discuss how this transformation stage often only affects the quantitiza- 
tion of the data, where qualitative data are quantified to then generate statis- 
tical analysis, and in fact “far fewer examples exist of the transformation of 
quantitative data into qualitative data” (Creswell & Plano Clark, 2007, p. 140). 
A further problem with the transformation of data is that often in MMR the 
number of individuals in the quantitative and the qualitative datasets are far 


from equal and yet to make any direct comparison some balance would be 
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necessary. In this study, 151 respondents completed the first questionnaire 
and yet there were only 10 EMI lecturers interviewed, so transforming the 
data into one dataset or the other would have led to an unbalanced analysis 
of the participants’ perceptions. 

In order to merge the data, therefore, I compared the datasets and dis- 
cussed the similarities and differences between the quantitative and qualita- 
tive results, which can be found in Section 6.3. This approach is known as a 
“side-by-side comparison” (Creswell, 2014, p. 222) and is commonly used in 
convergent parallel mixed methods design. 


5. Data Analysis 
5.1 Data Analysis Framework 


Given that this research collected both quantitative and qualitative data and 
that they were analysed over the same time span due to the convergent paral- 
lel mixed methods design of the first phase of the study, a clear analytical 
framework was required. The data analysis framework that was adopted in 
the initial AR cycle to analyse the needs of the students and inform the first 
modifications to the syllabus comprised two frameworks, one for the qualita- 
tive data analysis and one for the quantitative data analysis. For clarity, these 
two frameworks have been visually represented side by side in this volume, 
which is a typical way to represent data analysis in mixed methods research 
(Bryman, 2006; Riazi, 2017). However, it is important to note that the stages in 
the framework did not necessarily take place at exactly the same time, nor are 
they in any sense comparable with each other from a theoretical perspective. 
The qualitative data analysis concerned the semi-structured interviews with 
the EMI lectures and the quantitative data analysis focused on the question- 
naire responses from the first questionnaire conducted in May 2014. 

In order to analyse the qualitative data, the eight-step coding process ac- 
cording to Tesch (1990), as summarised in Figure 5.1, was the sequence that 


was used. 
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1. Get a sense of the whole. Read all the transcriptions carefully. Perhaps jot down 
some ideas as they come to mind. 

2. Pick one document (i.e., one interview) — the most interesting one, the shortest, 
the one on the top of the pile. Go through it, asking yourself, "What is this 
about?" Do not think about the substance of the information but its underlying 
meaning. Write thoughts in the margin. 

3. When you have completed this task for several participants, make a list of all 
topics. Cluster together similar topics. Form these topics into columns, perhaps 
arrayed as major topics, unique topics, and leftovers. 

4. Now take this list and go back to your data. Abbreviate the topics as codes and 
write the codes next to the appropriate segments of the text. Try this prelimi- 
nary organizing scheme to see if new categories and codes emerge. 

5. Find the most descriptive wording for your topics and turn them into catego- 
ries. Look for ways of reducing your total list of categories by grouping topics 
that relate to each other. Perhaps draw lines between your categories to show 
interrelationships. 

6. Make a final decision on the abbreviation for each category and alphabetize 
these codes. 

7. Assemble the data material belonging to each category in one place and per- 
form a preliminary analysis. 


8. If necessary, recode your existing data. 


Fig. 5.1 — Eight-step coding process (based on Tesch, 1990) in Creswell (2014, p. 198). 


The data analysis framework for the qualitative analysis adopted for this 
study largely adhered to the above model as initially it involved affixing in 
vivo and descriptive codes (phases 1 and 2 in my research). Categorising was 
undertaken through axial coding (phases 3 and 4 in my research) and then the 
number of categories were condensed to the three superordinate categories 
through selective coding (phase 5 in my research). The phases can be seen in 
Figure 5.2 in the left column. 


91 


Integrating Classroom-Based Negotiation Into a Syllabus 


Qualitative data Quantitative data 
Interviews with lecturers Questionnaires from students 
(May 2014) 
Phase Phase 

1 Transcribing interviews with 1 Completed questionnaires trans- 

EMI lecturers ferred to Excel file and coding 
frame applied 

2 Open coding: affixing in vivo 2 Data transferred to SPSS and 
and descriptive codes to tran- further coding frame applied to 
scribed interviews and put- SPSS data; codebook created 
ting notes in the margins 

3 Axial coding 1: looking at all 3 Data screening and cleaning 
the coded transcripts and 
finding links between the data 

4 Axial coding 2: Categorising 4 Data manipulation 
through conceptual frame- 
works: identification of super- 
ordinate categories 
concerning what language 
skills and proficiency are 
required by students; 
problems students have with 
English in class; problems 
lecturers have in class 

5 Selective coding: generation 5 Exploring the data: descriptive 
of “core category” (Dörnyei analysis of questionnaires from 
2007, p. 61) and subcategories students: identification of 
of language use and profi- general problems 
ciency; subcategories of stu- 
dents’ language problems and 
strategies implemented by the 
lecturers and/or students 

6 Representation of results 6 Representation of results 

i Merged data 
7 Merging of QUAL and QUAN data and comparison: identification of categories 
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concerning what language skills and proficiency are required by students; prob- 
lems students have with English in class. 


Sorting of data from interviews with lecturers and the questionnaires with stu- 
dents to identify any links between the data 


Data Analysis 


9 Mapping the data: translating them into visuals where possible/necessary and 
theorising the findings 


10 Theory to practice: integrating findings into a proposed syllabus in AR Cycle 2 


11 Evaluation stage of syllabus in AR Cycle 2: student end-of-course surveys and 
analysis 


12 Analysis of in-class worksheets + field notes collected in AR Cycle 3 


13 Evaluation stage of syllabus in AR Cycle 3: integration of data from further ques- 
tionnaires — analysis and comparison across all three questionnaires 


14 Evaluation stage of syllabus in AR Cycle 3: student end-of-course surveys and 
analysis 


Fig. 5.2 — Data Analysis Framework 


Regarding the analysis of the quantitative data, the procedure presented by 
Dörnyei (2003a, p. 2007) was used as a basis for the right-hand section of the 


data analysis framework and can be summarised in the following diagram 


Analysis — descriptive 
Analysis — inferential 


Because the procedure proposed by Dérnyei was adopted for this study, the 


represented in Figure 5.3. 


Data | Data 


Input 
> A 2 ; 
data screening manipulation 


Apply -5 Create 
coding frames codebook 


Fig. 5.3 — Quantitative data analysis sequence based on Dörnyei (2007) 


stages that have been represented in the diagram in Figure 5.3 map exactly the 
phases in the quantitative data analysis framework in Figure 5.2. 

Once the data had been explored, coded and displayed, the results that 
were produced were then merged following the procedure for convergent 
parallel mixed methods design as introduced in the previous chapter and 
which will be explained further in Section 5.6. 


5.2 The Data 


The raw data used in this study are summarised in Table 5.1. There were var- 
ious stages of data collection in the study due to the AR Cycles that were used 
and these are illustrated in the first column. The data analysis also occurred 
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at various stages during the study, starting with the initial data analysis in the 
needs assessment stage in AR Cycle 1. Further data analysis stages occurred 
in AR Cycles 2 and 3 following the collection of further data from the students 
predominantly as a means to evaluate the implementation of the various 


forms of the syllabus. The data collection methods, and timeframes for the 


collection and analysis can be seen in Table 5.1 below. 


Table 5.1 — The raw data 


AR cycle / Questionnaire | Interviews | In-class Field End-of- 
time worksheets notes So 
surveys 
1 / May - 151 completed | 10 EMI x x x 
August 2014 | (218 accesses) | lecturers 
2/Novem- | 99 completed x x x 63 (E&M) 
ber 2014 — (141 accesses) 
September 
2015 
3/ October | 115 completed | x 11 (PPE) + vv 15 (PPE) + 
2015 — (147 accesses) 27 (E&M) 90 (E&M) 
July 2016 
Total 365 10 38 2 sets 168 


There were two data collection methods in AR Cycle 1, as described in the 
previous chapter, and the data from these two methods — the questionnaire to 
the students and the semi-structured interviews with the EMI lecturers — were 
analysed progressively over time. The questionnaire to the students was 
online for a month and the results were then transferred into an Excel file im- 
mediately after the questionnaire time limit expired, which coincided with the 
period in which the interviews were being conducted. The transfer of the 
questionnaire data into SPSS, however, only took place later in the following 
academic year due to work commitments and the fact that the transfer of the 
data was predicted to be a long process due to the quantity of the data col- 
lected. 

The interviews with the EMI lecturers were conducted between May and 
July 2014, so some of them were conducted at the same time as the question- 


naire was online, and they were transcribed over the rest of the summer. The 
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transcribing process was a “time-consuming” (Dérnyei, 2007, p. 246) enter- 
prise as although there was only 2 hours and 20 minutes of interviews to be 
listened to, some parts of the interviews were challenging to transcribe due to 
the speed of speaking and the English used by the interviewees. However, 
transcribing them myself provided an opportunity for me to “get immersed 
in the data, an experience that usually generates emergent insights” (Patton, 
2002, p. 440). Although the initial coding of the interviews was only done in 
the following academic year, data analysis was effectively occurring from the 
very start of the data collection during the interviews themselves and again 
during the transcribing process. This process has been noted by Miles et al., 
who contend that “the qualitative analyst interprets what things mean by not- 
ing patterns, explanations, causal flows, and propositions” (2014, p. 13). While 
“research texts typically make a hard-and-fast distinction between data col- 
lection and analysis” (Patton, 2002, p. 436), the iterative nature of this study, 
as well as its duration, meant that data were collected and analysed at many 
different stages and not necessarily in such a linear manner as the data analy- 
sis framework or Table 5.1 may suggest. Miles et al. (2014, p. 12) further sug- 
gest that data analysis involves three “flows of activity”: data condensation 
(as they prefer to call data reduction), data display (the creation and use of 
instruments to present data, whether that means with text or with visuals), 
and drawing and verifying conclusions. These activities can often occur con- 
currently in research and data analysis, thus emphasising the iterative nature 
of qualitative research particularly. 

The data in AR Cycles 2 and 3 were mainly collected and analysed in close 
sequence: the end-of-course surveys were coded, the in-class worksheets 
were read in detail and the first set of field notes were typed up quite soon 
after being taken and were then analysed. The questionnaire data were pro- 
cessed and descriptive statistics were generated using SPSS as explained in 
Section 4.8.2 although the comparison of the three questionnaires only oc- 
curred at the end of AR Cycle 3. A detailed account of the data analysis in AR 
Cycles 2 and 3 is to be found in Section 5.6. 
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5.3 Analysing the Quantitative in AR Cycle 1 — 
The Questionnaire 


The questionnaire data collected in the first AR cycle of this study comprised 
151 completed answers. Table 5.2 below summarises the characteristics of the 


sample who completed the first questionnaire (n=151). 


Table 5.2 — Basic characteristics of respondents in the first questionnaire 


Characteristic % n 
Sex Female 72.8 110 
Male 27.2 41 
Age 18 0.66 1 
19 13.24 20 
20 16.55 25 
21 25.16 38 
22 16.55 25 
23 11.25 17 
24 7.94 12 
25 3.31 5 
27 4.63 Z 
30 0.66 1 
L1 German 39.07 59 
Italian 42.38 64 
Germantltalian 7.28 11 
Ladin 5.29 8 
Other 5.96 9 
Degree course PPE 21.85 33 
E&M 78.14 118 
Study year Ist 25.82 39 
2nd 33.77 51 
3rd 26.49 40 
“fuori corso” 13.90 21 
International Yes 53.64 81 
certificate No 46.35 70 
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As can be seen from Table 5.2, and as summarised in Prior (2021), almost 80% 
of the sample were following the E&M degree programme, and a third of them 
were in the second year. The course that I was teaching at the time the ques- 
tionnaire went online was the course for second-year E&M students, which 
could explain why more students from these two cohorts completed the ques- 
tionnaire, even if numerically there were more first-year students enrolled at 
the Faculty. The “fuori corso” label in the category titled “Study year” refers 
to students who have exceeded the statutory three years for an Italian under- 
graduate degree programme. As far as their first languages are concerned, the 
groups comprised an almost equal distribution between L1 Italian and L1 Ger- 
man, despite the Faculty having an overall distribution of approximately 70% 
Italian L1 students at the time of the study. The next L1 group, Ger- 
mant+tltalian, was included so that any local students who are bilingual would 
be represented, given the specific language context of South Tyrol, where the 
university is located. In South Tyrol approximately 3.8% of the population de- 
clare they are bilingual German-Italian speakers (ASTAT, 2015, p. 35) and so 
the category was pertinent for this context. The next language in the list is 
Ladin, one of the three official languages of South Tyrol together with German 
and Italian, and which comprises 4% of the South Tyrolean population 
(ASTAT, 2015, p. 22). The final group for L1 is “other” to account for any stu- 
dents who did not have the local languages as their L1. The languages repre- 
sented in this category were Ukrainian, Russian, Hungarian and Albanian. No 
students declared themselves to have English as an L1. Finally, just over half 
of the respondents stated they held one or more international English-lan- 
guage certificates, which ranged from the Cambridge English First to IELTS 
and TOEFL. 


5.3.1 Applying coding frames 

As explained previously, the data from these questionnaires were extracted 
from the Opinio questionnaire software and transferred via an Excel file into 
SPSS, given its user-friendliness and greater range of functions for the specific 
statistical analysis that was suitable for this study. The data that was to be 
analysed in SPSS had coding frames applied to the variables that had been 
selected, which initially were the variables from rows 1 to 20 in Figure 5.4 
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below. The coding frames for each variable were applied once a label had been 
provided for each variable and which constituted the codebook for the data, 
which was discussed in Section 4.8.2. Figure 5.4 shows the variable view that 
is produced by the SPSS programme showing the variable labels in the second 


column entitled “Name”. 


Values 
{1.00, time 1.. 
{1.00, PPE}... 
{.00, Yes}... 
{1, German}... 
{1.00.never}... 
{1.00.never}... 
{1.00.never}... 
{1.00,never}... 
{1.00, v diffic... 
{1.00, v diffic... 
{1.00, v diffic... 
{1.00, v diffic... 
{1.00, At}... 
{1.00, At}... 
{1.00, At}... 
{1.00, At}... 
{1.00, At}... 
{1.00, At}... 
{1.00, A1}... 
{1.00, At}... 
{1.00, once ... 
{1.00, once _. 
{1.00, once ... 
{1.00, once .. 
{1.00, Germ... 
{1.00, A1-B2.. 

Programme = 2... {0, Not Sele... 


Width Decimals 
2 
2 
2 
it) 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
2 
i) 


Fig. 5.4 — “Variable view” from SPSS 


The first variable in row 1 in Figure 5.4 is labelled “Time” and referred to each 
of the three timeframes when the questionnaires that were analysed during 
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this study were online, so the coding frames that were applied were 1, 2 and 
3 for the three years of the data collection and analysis. The second label is 
“Programme” and referred to the two undergraduate degree programmes 
that the students were following so it contained two values, PPE and E&M. 
“CertQu4” refers to whether the students had stated they had an international 
English-language certificate and coding frames were applied to comprise the 
two values corresponding to yes or no. L1 refers to the first language of the 
students. The next four variables refer to the frequency of practice of each skill 
that each respondent claimed they did, so “PracticeFreqR” corresponds to 
reading and so on. This was designed in the questionnaire as a rating scale 
and there were five choices: never, once a month, once a week, 2-3 times a 
week, every day, so the coding frame that was applied was 1-5 for the five 
responses. The next four variables refer to question 20 in the questionnaire 
that used a 4-reponse Likert-type scale to answer the question “How difficult 
do you find the following in English?”, referring to the four skills, reading, 
listening, writing and speaking and providing four response options — “very 
difficult, quite difficult, quite easy, very easy”. “DifficultyR” therefore refers 
to reading, “DifficultyL” to listening and so on. Row 13 to row 20 in Figure 5.4 
refer to questions 21—24 in the questionnaire which were “Level of competence 
in English. Please indicate your level (from + to ++++++)”, which had a differ- 
ent rating scale to the “DifficultyR” responses. For these responses, the coding 
frames 1-6 were used to refer to the language proficiency levels A1-C2 from 


the Common European Framework of Reference. 


5.3.2 Data screening 


Once the data were inserted, labelled and the coding frames were created, data 
screening was performed, which “involves spotting and correcting as many 
... errors and inaccuracies as possible before starting the analyses” (D6rnyei, 
2007, p. 202). As all the data were inserted manually into SPSS, mistakes did 
occur due to some values being inserted incorrectly. Dérnyei suggests the only 
way to detect these mistakes is to take the entire dataset and reinsert it into a 
new SPSS file, which he characterises as “a very laborious procedure” (2007, 
p. 202). For this study, however, I chose to verify the correctness of the data 
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inserted into SPSS by manually comparing them with the data that had al- 
ready been transferred and processed into Excel files by the questionnaire 
software. As this process had been performed automatically by the software, 
there were no inaccuracies in the Excel files. This alternative way of data 
screening was also laborious but allowed me to correct the inaccurately in- 
serted values. Dornyei refers to “contradicting data” as another cause of inac- 
curacies when data are being inserted into statistical software mainly due to 
routed items in a questionnaire (2007, p. 203). The questionnaires used in this 
study did contain routed items, which are questions that are only answered if 
a respondent gives a certain answer. For example, question 4 asked “Do you 
have an international language certificate?” and there were only two possible 
answers, yes or no. If respondents answered “yes”, they were routed to the 
next question which required them to select which certificate(s) they had 
taken, with the grade and year passed. If they answered “no”, they were 
routed on to question 6 requesting information about the in-house language 
exam used at the University, thus skipping the question about which interna- 
tional certificates they possessed. However, the questionnaire software al- 
lowed for this routing when the data were transferred into the Excel files and 


so there were no errors due to contradicting data. 


5.3.3 Data manipulation 


In Dérnyei’s sequence used for data analysis in Figure 5.3, the final stage be- 
fore the analysis itself is “data manipulation”, which he describes as 


making changes in the dataset prior to the analyses in order to make it more appro- 
priate for certain statistical procedures; it does not involve biasing the results one 


way or another. (2007, p. 204 original italics) 


Data manipulation was undertaken for certain variables due to the statistical 
analysis that was deemed appropriate for the data that had been collected. 
Therefore, as an example, rows 21-27 were subsequently added to the SPSS 
file to manipulate certain variables for analysis. Row 21 was labelled “Prac- 
ticeFreqR_2” and the initial coding frames of 1-5 that had been applied to the 
five choices, “never, once a month, once a week, 2-3 times a week, every day”, 


were changed as the first three responses contained fewer than five responses. 
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As this represented too few responses for meaningful analyses to be gener- 
ated, the first three responses were aggregated and recoded as 1, “2-3 times a 
week” was recoded as 2 and “every day” became 3. This procedure was rep- 
licated for all the variables referring to frequency of practice (rows 21-24). 
Row 25 contains the variable label “L1_ITDT” and refers to the L1 of the 
students excluding all the choices apart from Italian and German. Again, this 
was done in order to avoid having any values with fewer than five responses 
when the analysis was run. Row 26 shows the variable label “IcanS_2” and 
this was recoded to contain only three values where 1=A1-B2, 2=C1 and 3=C2. 
This was also done to avoid having cells with fewer than five responses when 
the analyses were run. Consequently, as can be seen above, data manipulation 
was necessary in this study in order to make the data analysis meaningful, 


which is described in the next section. 


5.3.4 Data analysis — Descriptive statistics 


he data that were collected from the students in the questionnaires sought to 
investigate various aspects connected to studying subjects where English is 
used as a medium of instruction. These data were also used as part of the 
needs assessment stage for the design and implementation of a new syllabus 
for the students’ ESP course that takes place in their second year of studies. 
The questionnaires sought to provide some answers to the first research ques- 
tion of this study, which aimed to discover what English-language skills are 
needed by economics students at this trilingual university. Therefore, one aim 
of the questionnaires was to collect data regarding effectively what skills stu- 
dents had to use in their studies at the Faculty. The questionnaires also sought 
to explore the frequency of practice of each skill and respondents were asked 
how often they engaged in practice of these skills. In this way these data could 
be analysed and then merged with the data collected from the EMI lecturers 
for triangulation purposes to ascertain what similarities or differences there 
were between the datasets (as from phase seven in the data analysis frame- 
work in Figure 5.2). 

The questionnaires also sought to elicit what students perceived to be 
their strengths and weaknesses in the four language skills, reading, listening, 
writing and speaking. The aim of these questions was to provide further data 
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to help answer the first research question and to understand which skills 
could then be focused on in the development of the syllabus given the rela- 
tively short course length of 30 hours. Therefore, various questions were in- 
cluded regarding whether they experienced any difficulties in the skills (ques- 
tion 20 in the questionnaires) and what they felt their proficiency level was in 
each skill and what level they felt they needed in each of the skills (questions 
21-24). The responses provided created ordinal data, which are used to place 
people in order on a scale. Therefore, students who selected 1 for speaking 
level of difficulty in question 20 can be said to find speaking more difficult 
than those who selected 4, which corresponded to the answer “very easy”. 
However, the difference between 1 and 2 on the scale cannot be said to be the 
same as the difference between 3 and 4 and so the intervals were not equal on 
the scale. Data that produce scales that have equal intervals, and which are 
placed in an order of magnitude, are known as interval or continuous data. 
Although interval data tend to produce more “precise” (D6rnyei, 2007, p. 227) 
information than nominal or ordinal data, the questionnaires were not de- 
signed to elicit this type of data due to the context and the research questions. 
The I can, I have to questions (questions 21-24) were also ordinal scales because 
the intervals are not equal on the scale, although, again, it can be said that 
response 5 is larger in magnitude than response 2. 

Moreover, the questionnaires aimed to investigate whether there was a 
relationship between how much practice students stated they did of a skill 
and their self-reported proficiency in that skill, in order to support the ra- 
tionale for the skills-based approach to the syllabus and to advocate an in- 
crease in the amount of certain skills work done in the course. This is known 
as hypothesis testing and so a test was needed to examine whether the varia- 
bles concerning students’ self-reported difficulty and proficiency levels in the 
skills were related in any way to the frequency of practice that they stated they 
did. 

The questionnaires also contained questions that were designed to elicit 
from the students specific information about their degree programme and 
study year, their L1, age and gender, and whether they had done any interna- 
tional English-language exams. These questions produced variables to be an- 
alysed that were nominal data, which “concern facts that can be sorted into 
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various categories” (Dérnyei, 2007, p. 228). Unlike ordinal data, nominal data 
refer to data where there is no order of magnitude, such as gender, since peo- 
ple can be placed into the two categories for gender but these categories can- 
not be ranked. Therefore, the data obtained from these questions are in the 
form of frequency counts. These nominal data were collected in order to assess 
whether there were any relationships between the variables used and the stu- 
dents’ self-reported proficiency and/or difficulty in the skills, so again hypoth- 
esis testing was being undertaken. These data about L1 and degree pro- 
gramme were also collected so that they could be compared with the data col- 
lected from the EMI lecturers so that the findings generated could be triangu- 
lated in the data merging phase. 


5.3.5 Pearson chi-square test 


One of the aims when designing the questionnaires was, as stated, to collect 
data for hypothesis testing, in other words to determine whether there were 
relationships between certain variables analysed. Therefore, in the analysis 
stage, a procedure needed to be used that would be able to identify whether 
there were significant relationships between these variables, thus demonstrat- 
ing that the variables were dependent. Moreover, since the variables were all 
nominal and ordinal data rather than interval or continuous data, a test was 
required that could be used with these data. The Pearson chi-square test, oth- 
erwise known simply as the chi-square test, is used to investigate relationships 
between variables, and “is one of the few procedures that can deal with nom- 
inal data” (Dörnyei, 2007, p. 228) and for this reason it was selected as the most 
appropriate test to use. The chi-square test is used in hypothesis testing, which 
considers the null hypothesis that states that “any observed pattern is due 
solely to chance and that, hence, no relationship exists’ (Berman & Wang, 
2017, p. 181). In statistics, the fact that no relationship exists, the null hypoth- 
esis, is assumed because there is always the concern that a relationship might 
appear in the sample taken for the study, but this relationship might only have 
appeared by chance. Therefore, one of the aims of running statistical tests is 
to ascertain whether the null hypothesis can be precluded. In this study, one 
aspect that was investigated was whether students’ self-reported proficiency 
in a skill had any relationship with the amount of practice they stated they did 
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in that skill. The null hypothesis in this case would be no relationship exists 
between skill proficiency and frequency of practice and the alternate hypoth- 
esis, which is the logical opposite of the null hypothesis, would be that there 
is a relationship between skill proficiency and frequency of practice. If there 
is a relationship between two variables, hypothesis testing will then test the 
significance of the relationship and a Pearson chi-square value is produced. In 
the social sciences the statistical standard of significance is often 5% or p < .05, 
which signifies that the variables are dependent and the relationship is signif- 
icant. Therefore, to test whether there was a relationship between skill profi- 
ciency and frequency of practice, cross-tab analyses were run in SPSS using 
the data related to the frequency of practice in the different skills and the re- 
spondents’ self-reported difficulties in those same skills in order to produce 
chi-square results. Tables were generated in SPSS for the skill of writing relat- 
ing to the full questionnaire data, in other words the results of the three ques- 
tionnaires combined (n=365) and the table below summarises the main find- 


ings from the analysis: 


Table 5.3 — Relationship between frequency of practice and self-reported difficulties regarding 
writing 


very difficult | quite difficult | quite easy very easy 
never 41.7% 50.0% 8.3% 0.0% 
once a month 9.5% 49.2% 38.1% 3.2% 
once a week 3.7% 43.1% 44.0% 9.2% 
eas R 7.0% 29.6% 51.3% 12.2% 
every day 7.4% 37.3% 44.4% 11.0% 


Reading along the rows, it can be seen that almost all of the students (91.7%) 
who stated they never practised writing found it “difficult” whereas a signif- 
icant proportion of students (55.4%) who practised every day found writing 
“quite or very easy”. 

Moreover as can be seen in the second table in Figure 5.5, the Pearson chi- 
square value, which is highlighted, is .000, in other words much less than 
p <.05, which signifies statistical significance. However, as mentioned, some 


of the cells contained fewer than five results, which is always identified by 
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SPSS (see second highlighted section in Figure 5.5) and so the data were ma- 
nipulated to remove this problem. As these cells with fewer than five results 
concerned the first three practice frequency variables, these were merged to 
produce three frequencies as can be seen in Table 5.4 in the first column: 


Table 5.4 — Relationship between frequency of practice and self-reported difficulties regarding 
writing — after data manipulation 


very difficult | quite difficult quite easy very easy 
once a week or less 8.2% 45.7% 39.7% 6.5% 
2-3 times a week 7.0% 29.6% 51.3% 12.2% 
every day 7.4% 37.3% 44.4% 11.0% 


Once the data had been manipulated, although the values for 2-3 times a week 
and every day remained the same, just over half (53.9%) of the students who 
stated they practised writing once a week of less found writing difficult. How- 
ever, the Pearson chi-square test that was generated with this new test still 
shows a significant p-value of 0.004. 

These analyses were run for all the skills to test for relationships between 
the variables. These cross-tab analyses were also run to test for relationships 
between frequency of practice and the responses to questions 21-24 where re- 
spondents were asked to rate their proficiency according to what they could 
do in each skill and what they felt they needed in the I can, I have to questions. 
Other cross tab tests were run to test for any significant relationships between 
the nominal variables concerning programme and L1 and the frequency of 
practice of the skills and respondents’ self-reported proficiency in the skills. 
The results of these analyses and discussion can be found in Section 6.2.1. 

The quantitative data, once analysed, were then merged with the qualita- 
tive data from the interviews, the analysis of which is detailed in the following 
section. The merging of the data is described in Section 5.6. 


5.4 Analysing the Qualitative Data in AR Cycle 1 — 
The Semi-Structured Interviews 


The spoken data in this study comprised the ten semi-structured interviews 
conducted with the lecturers who were using English as a Medium of 
Instruction at the time of the interviews (2013/2014 academic year) and which 
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were all recorded in their entirety. Table 5.5 on the following page presents 
basic details about the interviewees, whose names have all been changed to 
guarantee anonymity. However, their L1 and teaching subjects are real as they 
are relevant to this study. The analysis of the transcribed data involved coding 
and categorising and will be explained in the following subsections. 


Table 5.5 — Interview participants — basic data 


; ; Length of 
Alias Subject taught L1 ‘ 8 ; 
interview 

Interview 1 Dario International Finance Italian 10:35 
E ic Policy f 

Interview 2 Enzo A T Eey Italian 9:07 
PPE 

: y ; : Russian/ 
Interview 3 Ivan Financial Analysis ; 13:09 
Byelorussian 

Interview 4 Riccardo Financial Risk Italian 10:52 
Management 

Interview 5 Oscar Trin ciples of Italian 20:18 
Philosophy 

Interview 6 Claire Introduction to French 21:09 
Accounting 

. i Information Systems : 

Interview 7 Ottavio Italian 10:15 
and Data Management 

Interview 8 Fabio Economics Italian 12:34 

Interview 9 Benno Political Science 1 Italian 20:30 


jaeiew 10 Rodion Mathematics for Russian 10:00 
Economists A & B 


2 h 18 mins 
29 seconds 


5.4.1 Coding the interviews — Open coding 


Once the spoken data had been transcribed, the data were coded for the first 
time, corresponding with phase two in the data analysis framework (Figure 
5.2). As discussed in Section 4.8.1, categorical analysis was undertaken follow- 
ing a predominantly deductive approach due to the fact that an interview 
guide had been used, which aimed to analyse the target situation and investi- 
gate the problems lecturers felt they themselves and their students were expe- 
riencing when learning and teaching in English. Therefore, the open coding 
phase involved in vivo coding, where “words or short phrases from the par- 
ticipant’s own language” (Miles et al., 2014, p. 74) were used as codes when 
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referring to the answers to the target situation analysis that lecturers discussed 
in the interviews. Table 5.6 below shows some examples of in vivo coding 
taken from interview 6 with Claire, the French L1 lecturer of accounting. In 
the first column, there are extracts where she describes the skills used in her 
class. The skills she uses are typified by the first-person singular pronoun “I 


vou 


lecture”, “I ask questions” whereas the skills the students use are referred to 
using “they have to...” (these parts are in bold for clarity). 

Given the person-centred approach to this study, and therefore for onto- 
logical reasons, in vivo coding was also used in this phase as this type of coding 
is often used in “studies that prioritize and honor the participant’s voice” 
(Miles et al., 2014, p. 74). The in vivo coding was done on paper with different 
coloured highlighters and identified words and phrases that referred to prob- 
lems experienced by students and lecturers in the courses and how English 
was used in each lecturer’s course as expressed in the answers to questions 1 


and 2 in the interview guide. 


Table 5.6 — Examples of in vivo and descriptive coding for categorical analysis 


Skills used by the students 
and lecturers (orange) 


Lecturers’ 
problems (green) 


Students’ 
problems (pink) 


“T lecture, I ask questions 
from the students” 


“Er, so in class they have to 
listen... the course it’s all in 
English they have to 
understand. They have to do 
the reading, well that’s not in 
class. Sometimes I give them 
some... in my slides there is 
always some questions or 
some multiple choice things 
and they have to read it on 
the spot, think about it and 
then, you know, choose the 
right answer and then I show 
the right answer” 


“T have the same problem in 
speaking all languages is I 
lose my words” 


“It started when I first 
started being bilingual 
French English and so I see 
the sentence I build the sen- 
tence in my head and I see 
there’s a hole in it and I see 
it coming and the hole is 
there... and I panic I do par- 
aphrases and sometimes I 
just block and students help 
me find the right word. 
Sometimes it comes in the 
wrong language, even in 
German which I don’t 
speak” 


“Sometimes they 
will act as if they 
are struggling with 
the content but in 
fact they just don’t 
want to answer in 
English I think” 


“the written pro- 
duction usually I 
mean some of them 
are not very good 
but most of them 
are quite good and 
so but I see that 
they have they’re 
not confident with 
their speaking...” 
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Once the in vivo coding had been completed, descriptive coding occurred 
where descriptive labels were affixed to the interview transcripts to summa- 
rise the in vivo coded data. As categorical analysis was being used, these de- 
scriptive codes were actually the main categories that had been determined 
before the analysis by the interview guide questions, and there were three: 
“skills used by the students and lecturers”, “lecturers’ problems” and “stu- 
dents’ problems”. This was initially done manually but was then repeated us- 
ing the NVivo version 12 software where the transcripts were coded by affix- 
ing nodes to the data. As this was descriptive coding, phrases, sentences and 
even whole paragraphs were coded in this way. Although this coding only 
remained on the transcripts of the interviews and in the NVivo programme, 
Table 5.6 above summarises how this colour-coding was undertaken, using 


examples of in vivo coding from interview 6 with Claire. 


5.4.2 Coding — Axial coding — First attempt 


Through this initial coding process, however, it became apparent that besides 
the many different problems experienced by the students and the lecturers 
that were being described, certain other themes began to recur during the lec- 
turers’ accounts of their classroom experiences and the problems that arose 
during their course. This was emphasised by the fact that sometimes large 
sections of the interview transcripts had not been colour-coded, a phenome- 
non also reported by Lankshear & Knobel (2004, p. 271) in one of their own 
studies. Therefore, axial coding, where “the researcher makes connections be- 
tween categories, thereby attempting to integrate them and group them into 
more encompassing concepts that subsume several subcategories” (Dörnyei, 
2007, p. 261) was used to identify the axis of the three superordinate categories 
that emerged in this phase. Moreover the emerging categories and subcatego- 
ries were explored and connections were made (phase 3 in the data analysis 
framework). The terms main and superordinate are both used to refer to the 
principal category in coding, so for ease of explanation in this volume, main 
categories refer to the three categories that were identified in the open coding 
phase as described above, and superordinate will now be used to refer to the 
principal categories identified in the axial coding phases. 
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Axial coding was undertaken using a mixture of NVivo and hand-written 
mind-maps and notes where various conceptual frameworks were mapped 
out in an attempt to identify the superordinate categories and subcategories 
that were emerging from the interview data and how, and how far, they con- 
nected. A conceptual framework is “a visual representation of your main con- 
ceptual ideas about a study and how they interact and interplay with each 
other” (Miles et al., 2014, p. 24) and functions as a means to observe how the 
research is evolving. As such a conceptual framework is a means to display 
data, one of the three activities that Miles et al. identify as being integral to 
data analysis (see discussion previously in Section 5.3) and given the iterative 
nature of qualitative research, therefore, a conceptual framework tends to be 
relatively dynamic and will transform over the course of the study. 

This process of evolution was certainly present in this study. Although 
the interview guide had been used to direct the interviews and to try to make 
them more systemic and comprehensive, the lecturers were all teaching com- 
pletely diverse subjects with differing levels of English-language proficiency 
needed from themselves and from the students. Further, as clearly they were 
all individuals with their own experience and approach to teaching, and they 
all had a different level of English-language proficiency which also affected 
the data being provided, their “stories” (Patton, 2002, p. 10) differed widely. 
Consequently, although they were all asked fundamentally the same ques- 
tions, the answers varied depending upon their own experience and context. 
Using visuals to map out the conceptual frameworks in this phase of the cod- 
ing therefore facilitated the identification of the emergent categories. 


5.4.2.1 Superordinate category 1 — Classroom interaction 


The first superordinate category that emerged in this phase of axial coding 
was integrated into the following visualisation below that aimed to capture 
the “skills used by the students and lecturers” category that had been colour- 
coded orange in the open coding phase by creating a new superordinate cate- 
gory “Classroom interaction”. This new superordinate category also allowed 
the new category that emerged from the interviews, “strategies lecturers use 
to aid comprehension”, to be integrated as a subcategory, as can be seen be- 


low. 
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Classroom interaction 


Strategies lecturers use 
to aid comprehension 


What language skills 
students use 


What lecturers do 


emotional 
support 


practical 
measures 


writing 
(notes) 


speaking 


Fig. 5.5 — Data display during axial coding phase to visualise the “classroom interaction” 
superordinate category 

Although the main categories identified in the open coding phase had been 
generated deductively, the subcategory referring to the strategies lecturers 
used in class was generated inductively. Reference to strategies being used in 
class already appeared in the first interview with Dario, the lecturer for Inter- 
national Finance, as can be seen in the descriptive coding of the instance in the 


following example: 


Yeah so after the first time I understood that ! 1 strategy 
the only way to get them to understand some- 

thing was to first show all the footage and then 

talk about what was told and then show it 

again — piece by piece discussing what we had 

just heard — 


Through the data collection phase, therefore, it became increasingly apparent 
that the lecturers were spontaneously referring to the strategies that they used 
in class to facilitate learning and teaching without my prompting and in fact, 
every single lecturer referred to measures they took. This inductive coding, 
which is “better grounded empirically and [is] especially satisfying to the re- 
searcher who has uncovered an important local factor” (Miles et al., 2014, 
p. 81), effectively allowed me to discover significantly relevant data concern- 
ing the use of English in the lecturers’ classrooms and courses. Further, the 
use of the conceptual framework to map “classroom interaction” in the axial 


coding phase helped to illustrate how this emergent subcategory was linked 
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to the other subcategories concerning skills that had been coded deductively, 
and demonstrated again the iterative nature of the data analysis phases. 


5.4.2.2 Superordinate category 2 & 3 


There were two other superordinate categories that were developed during 
the axial coding phase: “Language problems” and “Lecturers’ approach to 
students’ language proficiency”. The “Language problems” category grew 
from the two other main categories that were coded in the open coding phase 
during a discussion with my supervisor. This meeting resulted in “peer check- 
ing” occurring, which involves “asking a colleague to perform some aspect of 
the researcher’s role — usually developing or testing some coding scheme” 
(Dérnyei, 2007, p. 61). Peer checking is a means to ensure that the data being 
generated is valid and was therefore one of the validity checks put in place for 
this study. 

The three superordinate categories that emerged during the axial coding 
phase, therefore, were: 

- classroom interaction 

- language problems 

- lecturers’ approach to students’ language proficiency 


The conceptual framework for the first category was discussed above and 
illustrated in Figure 5.7. A different conceptual framework was developed for 
superordinate category 2 and was represented in a table as can be seen in 
Table 5.7. 


Table 5.7 — The “language problems” superordinate category and its subcategories 


Supenonliiats Subcategories Characteristics 
category 
Students’ language | Comparisons (Italian vs. German) 
problems Language v content 
Adjectives referring to problems: 
Language “afraid” — “shy” — “handicapped” — 
problems “less able” — “(not) adequate” — 
“(not) confident” 
“they” + negative verb forms 
Lecturers’ Pronunciation (non-native accents) 
language problems | Technical language 
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The subcategories mirrored the main categories already identified in the open 
coding phase but were then integrated into one superordinate category, “Lan- 
guage problems”. The characteristics of the two subcategories emerged from 
the data that had been coded in vivo and descriptively in the initial coding 
phase and concerned recurrent themes. One of these themes was the perceived 
difference in language proficiency between the students with German and 
Italian as their L1 and particularly the view that Italian students had a lower 
proficiency than German students as in the following examples: 


Dario (1): Because I noticed that Italian students are actually less um — prepared in 
English and this is just my feeling than German background students are 

Enzo (2): Well the level of their English is is not let’s say it’s quite heterogeneous 
there are some students especially the German-speaking students who are really 
good I mean some students speak even better than me actually and probably the 
Italians have more problems and so I mean the level is quite heterogeneous actually 
but I guess that since I am Italian probably Italians can understand me because my 
English is with an Italian accent and so they probably no problems understating 
what I say and at the same time you know the German guys are good so 

Ivan (3): I see a big difference between German and Italian students. German stu- 
dents come here and they speak well almost perfect English when you have Italian 


students it’s always much lower level 


Moreover, there were references to the fact that the problems the students 
were experiencing were sometimes due to the language but often they were 
due to the content. In fact, this issue was mentioned by seven of the ten lec- 


turers, for example: 


Rodion (10): They have never had mathematics in English, first. Second, their 
knowledge of mathematics even in their national languages are not always good 
enough. That is it. 

Interviewer: So the problems they have, do you think they’re more problems with 
mathematics or is it problems with English? 


Rodion: It is both. 


However, the many references to problems with the content of the subjects, 
although they had been coded in the open coding phase, did not fit very well 
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into the “language problems” category. Therefore, this category needed to be 
revised. Miles et al. make the point that “some codes do not work; others de- 
cay” (2014, p. 82) and in this particular case it was the organisation of the cat- 
egory that did not seem to fit into the analysis. 

The characteristics referring to the students and their language problems 
were often expressed by the lecturers using adjectives, and this element 
emerged also due to the in vivo coding undertaken with NVivo. The software 
has a function that displays the frequency individual words appear and pro- 
vides either an exact match (e.g. talk) or a match with words that have the 
same stem (e.g. “talk” and “talking” are counted as the same word). Using the 
stemmed word frequency analysis helped to identify recurrent themes in the 
interviews. The figure below shows a word frequency analysis displayed as a 
tree diagram with some of the words that were coded indicated in the red 


circles. 
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Fig. 5.6 — NVivo tree diagram displaying word frequency 
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The programme also provides a count of the word frequency and the table 
below summarises the word count of each highlighted stemmed word shown 


in the previous figure over the ten interviews: 


Table 5.8 — Number of occurrences of certain in vivo coded words 


Word No. of occurrences 


Confident 
Lack 

Shy 

Less 
Afraid 


wr DD O 


These features of NVivo therefore allowed me to identify recurring themes 
and patterns but also allowed me to “[make] connections between categories, 
thereby attempting to integrate them and group them into more encompass- 
ing concepts that subsume several categories” (Dörnyei, 2007, p. 261) in this 
axial coding phase. 

The third superordinate category that emerged during axial coding was 
“Lecturers’ approach to students’ language proficiency” and this category was 
an attempt to capture the lecturers’ reflections about their courses and the stu- 
dents’ language proficiency and problems. It comprised characteristics such 
as the ways lecturers approached the fact that they were teaching their sub- 
jects using EMI and an example that was descriptively coded was “language 
mistakes not penalized”. However, this category was much less well defined 
than the other two superordinate categories and it contained references to the 
exams that each lecturer held for their courses, which did not necessarily re- 
late well to the superordinate category. Therefore, like the category referring 
to the students’ problems with the content of the courses, this also needed to 
be revised. 


5.4.3 Coding — Axial coding — Second attempt 


Given the fact that some of the categories established in the previous phase of 
coding did not seem to be fitting together effectively, I decided to suspend the 
coding. Coffey & Atkinson acknowledge that coding might not necessarily 
follow a neat, sequential process and state “analysis is not about adhering to 
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any one correct approach or set of right techniques: it is imaginative, artful, 
flexible and reflexive” (1996, p. 10). K. Richards agrees and states that the re- 
searcher might need to “make time to stand back and find different ways of 
seeing the data” (2003, p. 269). Therefore, after a couple of months I revisited 
the interview data in an attempt to revise the categories identified in the first 
axial coding phase. This next phase corresponds to phase 4 in the data analysis 
framework in Figure 5.2. The data were revisited using N Vivo and all the in- 
terviews were coded again using in vivo and descriptive coding as occurred 
in the open coding phase, but conceptual frameworks were being created sim- 
ultaneously in an attempt “to find alternative approaches to organisational 
and interpretive challenges” (K. Richards, 2003, p. 269). Therefore through this 
approach, three revised superordinate categories emerged: 

- use of English 

- problems encountered in the target situation 

- lecturers’ approach to language difficulties 


5.4.3.1 Superordinate category 1: Use of English 


The first superordinate category, “Use of English” attempted to summarise how 
English was used in the classroom but also in the exam for each of the lecturers’ 
courses. In this way, it presented information about the target situation, as the 
interview guide had been designed to do, and addressed the first research 
question of this study, “What are the English-language skills needed by eco- 
nomics students at this trilingual university as perceived by the main ‘actors’, 
i.e. students and lecturers?” It also presented data that could be directly com- 
pared with the data collected from the questionnaire to the students, who had 
been asked to provide details concerning what exactly they had to do for their 
EMI courses for each skill. Because neither the quantitative nor the qualitative 
data had been transformed as is often the case in mixed method designs (Cre- 
swell, 2015; Creswell & Plano Clark, 2007) and as discussed in Section 4.8.3.1, 
the “Use of English” superordinate category was developed in this way to make 
the merging of the quantitative and the qualitative data more meaningful for 
triangulation purposes. The first superordinate category that evolved in the se- 
lective coding phase is represented in the conceptual framework in the figure 
below. 
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As can be seen, the superordinate category allowed for the inclusion of the use 
of English in both the classroom and in the exam, the two contexts that were 
important for the lecturers’ courses. It also comprised the skills that the 
lecturers stated were necessary for the students to use in both the exam and 
in the classroom. The skills used in the classroom had already been coded as 
noted previously, but they were recoded to account for the difference between 
the language skills used and the academic skills used and as there was refer- 
ence to the study aids used in the classroom, this was included as a distinct 
subcategory of the use of English in the classroom. Therefore the initial in vivo 
coding that was undertaken in the open coding phase was re-examined and 
the word frequency function in NVivo was used and then the words were re- 
coded to distinguish between the two subcategories of “Language skills” and 
“Academic skills”. The word frequency table can be seen below in Table 5.9. 


Table 5.9 — Characteristics of the “in the classroom” subcategory and number of occurrences of 
coded words 


Word in vivo No. of 
: Code 
coding occurrences 
“ask questions” 27 + 26 Language skill 
“reading” 17 Language skill 
“talking” 16 Language skill 
“interact” 14 Language skill 
“slides” 14 Study aid 
“discuss” 11 Language skill 
“presentation” 11 Academic skill / 
Study aid 
“notes” 10 Academic skill 
“writing” 10 Language skill 
“exercises” 9 Academic skill 
“texts” 9 Study aid 
“pages “ 9 Study aid 
“speaking” 8 Language skill 
“books” 6 Study aid 
“assignments” 6 Academic skill 
“listen” 6 Language skill 
“powerpoint” 6 Study aid 
“video” 6 Study aid 
“anderstand” 4 Academic skill 
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The most frequent words that were coded were “ask” and “question” with 27 
and 26 occurrences respectively. Although these two words referred to the 
lecturers asking questions to the students, rather than what the students 
themselves had to do in class, this implied that students would have to answer 
the questions either orally or as written answers, and so these were included 
in the examples of the language skills needed by the students. The language 
skills were visualised in the conceptual framework in the order of frequency 
that they occurred, thus the order speaking — reading — writing — listening. 

The academic skills coded were “presentations, notes, exercises, assign- 
ments and understand”. Note-taking is such a common activity in university 
settings that it has been suggested that it is the “primary means of learning 
content” (Boyle & Forchell, 2014, p. 9) and so its frequency in this context is 
not surprising. Exercises, although not necessarily specifically defined by the 
lecturers, tended to be mentioned by the lecturers teaching the more mathe- 
matical subjects that were taught in English, and as such were coded “aca- 
demic” rather than “language skills”. “Assignments” were mentioned by two 
lecturers, Claire and Benno. While Claire’s assignments were written tasks, 
Benno referred to them as reading tasks. “Understanding” is a key academic 
skill in lectures and has actually been classified as one of the main academic 
problems that students experience when attending lectures (James, 1977 
quoted in Jordan, 1997), and as “comprehension difficulties may be com- 
pounded by insufficient knowledge of the specialist subject” (Jordan, 1997, 
p. 179), it was coded as an “academic skill”. 

The words referring to the study aids used in class and in the exams were 
also coded at this stage and as can be seen, the most frequent references were 
to “slides” and “PowerPoint”, “texts” or “books” and “pages”, thus implying 
the students had to practise a significant amount of reading in their classes. 
Although “presentation” was coded eleven times, most references to making 
presentations were to the lecturers themselves using presentation software ra- 
ther than students having to give presentations in class. Therefore, it was 
coded as both an academic skill for when students had to give presentations 
in class and as a study aid to be combined with slides and PowerPoint. There 
were also six occurrences of “video”, but these were only mentioned by two 


of the lecturers, Dario and Ottavio. Dario referred to videos sourced from the 
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Financial Times and the Economist news websites that he used in class (inter- 
view 1) whereas Ottavio stated “I produce videos so they will relisten to my 
voice recorded in the video”. 

The other main category in this superordinate category was the use of 
English “in the exam”. This category was subdivided into the three character- 
istics that emanated from the interview data: “language skills”, “vocabulary” 
and “question types”. Only one language skill was mentioned, and this was 
writing, given that all the exams at the Faculty are written. Vocabulary was 
mentioned by several lecturers and comprised references predominantly to 
the use of technical language both in the exam questions and in the answers 
required from the students. The final characteristic of the use of English in the 
exams referred to the type of questions used in the exam. Some lecturers ex- 
plicitly mentioned that their exams were almost completely number-based, 
given the proliferation of financial and mathematical subjects taught in Eng- 
lish at the time the data were collected, but when answers were required to be 
written, these questions were coded as mainly “short”. Only Claire (interview 
6) referred to anything longer than a short text, which in her case was a busi- 
ness report that had to be completed as an assignment, so therefore not in 
exam conditions. 

Consequently, this first superordinate category, entitled “Use of English”, 
aimed to capture all references the lecturers made to how English was used in 


their courses, both in their exams and in the classroom. 


5.4.3.2 Superordinate Category 2: 
Problems encountered in the target situation 

The second superordinate category that emerged in this second axial coding 
phase was “Problems encountered in the target situation”, which subsumed 
the “Language problems” category in the first axial coding phase. It also inte- 
grated all the other non-language related problems that had emerged in the 
various interviews but which I had not been able to incorporate satisfactorily 
in the categories created in the previous coding phase. 
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As can be seen from the figure on the previous page, this superordinate 
category can be considered the most complex category of the three 
superordinate categories as it concerns all the problems that the lecturers 
deemed were specific to their contexts. The “Language problems” maintained 
its distinction between the problems lecturers said they faced and the 
problems they identified the students faced, as had been categorised in the 
first axial coding phase, but a further subcategory was added, “no problems”. 
Two lecturers very explicitly asserted that they themselves did not have any 
language problems and one felt that the students did not have any problems. 
Therefore, although the interviews were designed to investigate the language 
problems that were present in the EMI classrooms, this subcategory was 
added to reflect the reality of some of the contexts under examination in order 
to “take account of all available evidence including discrepant cases” (K. 
Richards, 2003, p. 270). The lecturers’ language problems were further broken 
down into two subcategories given that all the instances of language problems 
they described either originated from their not being native speakers of 
English, or due to aspects related to the content of what they were teaching. 
The students’ problems the lecturers described, as can be seen in Figure 5.10, 
were considerably more diverse and numerous than those the lecturers 
identified for themselves and ranged from problems regarding vocabulary 
and skills to cognitive and affective problems that interlinked with the 
“content problems” and “classroom management” categories. 

Although many of the problems the lecturers described referred specifi- 
cally to language problems due to the EMI context, they continually referred 
to many other problems, thus the two other subcategories that were coded as 
“content problems” and “classroom management”. “Content problems” had 
been identified in the open coding phase, and were unable to be integrated 
satisfactorily in the first axial coding phase, but the development of the “Prob- 
lems encountered in the target situation” superordinate category enabled 
them to be incorporated perfectly. The content problems included aspects con- 
cerned with students not understanding due to lack of subject knowledge of- 
ten due to insufficient studying, such as in this example from Claire, who 
stated: 
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it’s not because they don’t know English. It’s because they haven’t studied. 
Whether they haven’t studied because it’s too difficult because it’s in English is an- 


other matter but I can’t go that far. 


However, problems regarding the content of the lectures did not only arise 
from lack of subject knowledge. Ottavio, the lecturer in Information Systems 
and Data Management (interview 7) stated the following: “Others are atten- 
tion problems, so they don’t pay enough attention to what I write or they don’t 
have any idea that that thing may be important”. As this statement was clearly 
referring to cognitive problems rather than problems concerning the subject 
knowledge, a further subcategory was added to include cognitive problems, 
which was positioned in the conceptual framework to show that it arose from 
the content but also that it was a problem some students were experiencing. 
Another recurring theme that again had been coded in the open coding 
phase but had not been subsumed into a superordinate category referred to 
problems linked to classroom management. Five of the lecturers made explicit 
reference to the size of the classes; some in a positive way if they had small 


class sizes as in the example from Enzo (extract no. 2): 


let’s say classical lecture but since as I said students was like very small class so 
there was a kind of interaction so I don't know what you mean by seminars but it 


was quite interactive let's say 


However, if class size was mentioned, it was predominantly due to the 
problems created by large class sizes such as Dario (extract no. 1) who af- 
firmed “it would be easier of course if there were fewer students”. Claire also 
raised the issue of large class sizes and the direct effect it had on the students’ 
speaking skills when she stated that “the fear of speaking out is especially in 
third year and first year with 100 students”. Fabio, the lecturer for the first 
year core Economics course referred to the large class size and how that might 
affect the students’ willingness to contribute in class and was in vivo and de- 


scriptively coded as below: 
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Fabio (8): This year for instance I had a very low 1 “shy” 
responses to I don’t know what happened, ! they 
were quite shy. 


Interviewer: So you have big classes, yeah? 


Fabio: 2 Yes, I teach to the biggest course of eco- 2 class size 
nomics students which is about 150 students. I 
used to have 60 even 60-70 students in class. Per- 
haps they are very, you know... 


Interviewer: So you think they didn’t answer or 
they weren’t so chatty let’s say because maybe they 
are shy in the end? 


Fabio: 3 Yes, yes 3 class size > 
shyness 


Fig. 5.9 — Coding showing two categories in one extract 


The “classroom management” subcategory also contained the further subcat- 
egories of “type of class” and “time problems”. The “type of class” subcate- 
gory attempted to group aspects concerning the style of lecturing used, which 
was variously described as “old-style”, “formal”, “frontal”, “classical” and 
“face to face”. Although these descriptions do not necessarily imply problems, 
they were inextricably connected to the class sizes, which did create problems, 
especially affective problems for the students and therefore the double arrow 
connecting class size to the type of class and affective problems in the concep- 
tual framework. The other aspect that was included in the “type of class” sub- 
category was an attitude expressed by Claire, who stated that “the room is 
horrible”. I felt such a strong comment needed to be included in the category 
given that it has been acknowledged that “the formal physical environment in 
which students take their courses has a significant impact on measurable stu- 
dent learning outcomes” (Brooks, 2011, p. 719). The final subcategory in this 
section was “time problems”, which was referred to by several lecturers with 


the common complaint that “I don’t have time”. 
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5.4.3.3 Superordinate category 3: 
Lecturers’ approach to language difficulties 

The third superordinate category that emerged from this second axial coding 
phase was the “lecturers’ approach to language difficulties” category that 
evolved from the initial “classroom interaction” category and which had 
mainly concerned the strategies that lecturers stated they used to aid compre- 
hension in their EMI classrooms. Figure 5.12 shows the conceptual framework 
that was developed to represent this superordinate category: 
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Fig. 5.10 — Lecturers’ approach to language difficulties superordinate category conceptual 
framework 


The approaches that the lecturers stated they used to mitigate language diffi- 
culties in the classroom included providing support material that ranged from 
translations of texts and glossaries of specialised vocabulary in three cases to 
providing past exam papers. Many identified ways they would employ lan- 
guage to assist students who were having difficulty in the classroom. Two of 
the lecturers stated they spoke “very slowly” if they realised students had dif- 
ficulty following their class, whereas Dario stated he tried “to mimic the Brit- 
ish accent and not talk too much like an Italian” given the number of German 
L1 students in his class. Others identified that complex language was often a 
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hindrance to comprehension and would use simpler language or explain un- 
known words (Ottavio and Fabio). Others used strategies that were less con- 
nected with purely language aspects and so were categorised as “teaching 
methodology”, which included encouraging the students to ask questions 
(Benno), providing examples where necessary (Dario and Ottavio), providing 
longer wait time for students to formulate answers (Ivan, Claire and Ottavio) 
and nominating students stronger in English to answer (Claire). 

Strategies used to manage difficulties in the exam were less numerous 
than those used in the classroom but the main one was the subcategory that 
had been descriptively coded as “language mistakes not penalized” in the first 
axial coding phase. With the revision of the superordinate categories and their 
subcategories in the second phase of axial coding, the “language mistakes not 
penalized” subcategory that had been much less well defined was now assim- 
ilated into the revised “Lecturers’ approach to language difficulties” superor- 
dinate category and was relabeled “not taking language mistakes into consid- 
eration”. The strategy of lessening the impact of students’ language difficul- 
ties on their exam performance by actually ignoring the mistakes had been 
mentioned by six of the ten lecturers and its inclusion was therefore funda- 
mental. 

The “Lecturers’ approach to language difficulties” superordinate cate- 
gory further evolved through the addition of two more subcategories which 
despite being coded in the open coding phase, had found no satisfactory po- 
sition in the first categories that developed in the first phase of axial coding. 
These subcategories were coded “where lecturers think students can im- 
prove” and “taking no action”. The first of these subcategories referred to 
comments from six of the ten lecturers who reflected on what aspects would 
improve either the students’ performance in their classes or the teaching of 
the subjects themselves. These aspects referred exclusively to either language 
skills or academic skills, including a remark from Benno, the lecturer of Polit- 
ical Science who stated 


I would suggest one more course of English for academic writing or ... so it could 
be also academic speaking or a scholarly... how to express yourself in a scholarly 


fashion 
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Although others (Oscar and Ottavio) also referred to students needing to im- 
prove their writing skills, Benno’s focus on academic writing was unique. 
However, his Political Science course was the most language-heavy of the sub- 
jects apart from Philosophy, and so this attitude is certainly understandable. 

The “taking no action” subcategory was included in this superordinate 
category because several lecturers referred to the fact that their primary focus 
was the subject they were teaching and the students’ language difficulties 
were not of their concern. Dario states “regarding students’ performance in 
English I mean I don’t really care much about it because the course is about 
finance” and Rodion states “it’s not my business” with regard to the language 
difficulties experienced by his students. 

The second approach to axial coding therefore not only clearly identified 
the main themes that had emerged from the interview data but enabled me to 
group them satisfactorily into three superordinate categories, which served as 
the basis of the selective coding phase that followed. 


5.4.4 Coding — Selective coding 


The last phase of coding took place soon after the axial coding phase and was 
the selective coding, which is represented as phase five in the data analysis 
framework. Selective coding is the phase where a central category is selected 
and which acts as the focal point of the study and connections are made be- 
tween the existing categories. Dérnyei states that because as much of the data 
as possible needs to be integrated, “the central category/theme needs to be of 
a sufficiently high level of abstraction to be able to subsume other categories” 
(2007, p. 261) and in order to achieve that, the ideas that are explored in the 
axial coding phase should be further developed. In this study, therefore, the 
main core category was produced in the selective coding phase by “identify- 
ing just one of the coded phenomena or themes that seem central in the study” 
(Gibbs, 2007, p. 87), the first superordinate category, and then integrating the 
other superordinate categories that emerged from the interviews with the EMI 
lecturers through the open and axial coding phases as explained in the previ- 
ous section. The analysis of the interviews was part of the needs assessment 


that took place in AR Cycle 1, which aimed to answer the first research ques- 
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tion of this study. Therefore, the core category that was produced in the selec- 


tive coding phase aimed to provide a clear answer to the research question 


and so the first superordinate category was recoded as “Use of English at the 


Faculty of Economics”. The core category is illustrated below in the concep- 


tual framework that sought to combine it with the other two superordinate 


categories. 
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Visually, the core category is placed at the top of the conceptual framework to 
emphasise its importance for this study as well as its importance in the context 
under study where English is used as a medium of instruction but is seldom 
the L1 of the students or the lecturers using EMI, which, as stated previously, 
differentiates English from German and Italian at the Faculty. This core cate- 
gory includes the two main subcategories that emerged in the axial coding 
phase, which identified the contexts in which English is used, and the second- 
ary subcategories detailing how it is used. The core category has been placed 
in this initial position not only because of its crucial role in this study and in 
the Faculty but also “because we need such a focal point to be able to bring 
together the other categories in a coherent manner” (Dornyei, 2007, p. 261). 
In order to incorporate all three superordinate categories identified in the axial 
coding phase, therefore, the core category is placed above the other two su- 
perordinate categories but they are all joined together to demonstrate the in- 
terconnected relationships between these three categories. With this arrange- 
ment, and the use of the connectors between the categories in the conceptual 
framework, the coherence between all three superordinate categories is 
shown. 

The core category, “Use of English at the Faculty of Economics”, is con- 
nected to the “Problems encountered in the target situation” since it was often 
the use of English itself that led to various problems in the target situation, 
such as students’ inability to understand the lecturers in class, or problems 
with unfamiliar vocabulary, as discussed. However, the conceptual frame- 
work above aims to visualise not only how the use of English in the target 
situation leads to problems for both the students and the lecturers that are of 
both a language and a content nature, but it also aims to capture how the spec- 
ificity of the target situation had an impact on the classes where EMI was im- 
plemented. In other words, as has been described previously in this chapter, 
some of the lecturers referred to how using EMI contributed to problems re- 
lating to how they taught their subject’s content. This was often due to the 
perception that the students’ English-language proficiency level was low or at 
least insufficient for the subject, or because the lecturers themselves felt they 
experienced problems with English or perhaps were not aware of methodo- 
logical strategies that could assist in EMI contexts where most students do not 
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have English as their L1. Others concentrated on how the subject’s content 
matter or the classroom environment contributed to language problems, often 
also due to affective issues where students proved to be reluctant to contribute 
in class. Therefore, this symbiotic relationship is captured by the €> label join- 
ing the core category to the “Problems encountered in the target situation” 
superordinate categories. The “problem ® strategy” label joining the other 
superordinate category, “Lecturers’ approach to language difficulties” with 
the “Problems encountered in the target situation” superordinate category 
aims to capture the problem/solution relationship between the two. 

The core category is also connected to the other superordinate category, 
“Lecturers’ approach to language difficulties”, and labelled with the double- 
headed arrow € to emphasise the two-way relationship between the two cat- 
egories. The approaches lecturers used to manage the various language diffi- 
culties are a direct consequence of using EMI (as opposed to the students’ L1) 
for the courses, and the approaches the lecturers used are also part of the use 
of English in the courses. 

The importance of the language skills and their use in the EMI courses 
has also been highlighted by placing it in the centre of the conceptual frame- 
work. This central position emphasises how the language skills that are 
needed for the courses, both in the classroom and the exam, are also a source 
of problems for the students. However, language problems were also seen to 
affect the skills, often due to affective aspects, as explained previously. There- 
fore, the €> label is used to signify the dual relationship between the two sub- 
categories. 

The conceptual framework that illustrates the core category with the su- 
perordinate categories placed around it in Figure 5.13 has been compressed 
and only contains two levels of subcategories branching from each superordi- 
nate category. Consequently, as can be seen in the enlargement of a part of the 
conceptual framework in Figure 5.14 below, the “ problems encountered in the 
target situation” superordinate category has the subcategory “Language 
problems” and the characteristics of that subcategory “students’ problems”, 
“no problems” and “lecturers’ problems”. The examples that were illustrated 
in the conceptual framework in Figure 5.10 are no longer represented. This 
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compression has been done purely due to space reasons and ease of illustra- 
tion and each superordinate category is to be understood to contain all the 


subcategories, characteristics and examples as illustrated the previous figures. 


Content 
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Fig. 5.12 — Enlargement of a part of the core category conceptual framework 
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Once the core category had been produced from the interview data, and had 
been visualised in the conceptual framework, it could be merged with the data 
obtained from the questionnaires for triangulation purposes following the 
convergent parallel mixed methods research design that characterised this 
phase of the study, corresponding to Phase 7 in the data analysis framework 
in Figure 5.2. 


5.5 Merging the Data 


The two sets of data that were collected and analysed in AR Cycle 1 of this 
study aimed to provide insights into the use of English in courses that were 
taught in English at the Faculty as a means to conduct a needs assessment and 
analysis for the development of the blended syllabus for the ESP course. These 
data were also analysed in order to provide answers to the first two research 
questions of this study. Therefore, these data were to be merged to determine 
to what extent the qualitative results confirmed the quantitative results as can 
be seen in Phases 7-9 of the data analysis framework (Figure 5.2). As explained 
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in the previous chapter, there are various ways to merge the data in mixed 
methods research, and an approach that is commonly used in convergent 
parallel mixed methods design, and therefore for this study, is using a “side- 
by-side comparison” (Creswell, 2014, p. 222) where the findings from the 
quantitative and the qualitative analysis are compared. This is usually 
undertaken either using a side-by-side joint display table, which presents 
“both qualitative themes and quantitative statistics results side by side in a 
table” (Creswell, 2015, p. 85) or using discussion. Given the complexities of 
the conceptual frameworks developed for the qualitative analysis and the 
amount of data analysed from the questionnaires, a side-by-side joint display 
table was considered inadequate to represent all the relevant data effectively, 
although the conceptual frameworks developed for the qualitative analysis 
were still used. Therefore, the results from the two datasets were compared 
and the discussion of these results can be found in Section 6.3. Once the data 
had been merged, the first redesigned syllabus that was implemented in AR 
Cycle 2 was created (phase 10 in the data analysis framework, see Figure 5.2), 
and this is also discussed in the following chapter. 


5.6 Analysing the Other Collected Data 


The following sections will now describe the analysis of the other data 
collected during the study: the end-of-course surveys that were conducted 
with the students at the end of the two academic years to evaluate the imple- 
mented syllabus, and the in-class worksheets and field notes that were used 
twice in class. There will also be a discussion of how the data from the second 
and third questionnaires administered to the students were compared with 
the initial questionnaire. The findings from these analyses will be presented 
in the following chapters. 


5.6.1 Analysing the data in AR Cycles 2 and 3 — 
The end-of-course surveys 


5.6.1.1 The first end-of-course survey 


Data collection occurred at various stages during this study after the initial 
needs assessment and analysis in AR Cycle 1. Further quantitative data were 
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collected, as has been described in Section 4.7, in the form of two end-of-course 
surveys, which were conducted during the written exams at the end of the 
two courses. The first survey was conducted since only two students had 
opted to follow the newly designed syllabus that was used in AR Cycle 2, and 
all the other students chose to follow the existing syllabus. Therefore, it was 
crucial to understand why there had been such a low take-up of the new 
syllabus so that adjustments could be made for the following academic year. 
As this was an action research study, reflection about the practice is an integral 
part of the whole process and this “reflection on practice is what gives rise to 
further ideas for practice” (Burns, 2010, p. 142). The end-of-course survey was 
therefore administered not only as a means to understand and reflect on the 
low participation rate but also to provide input for a revised version of the 
syllabus in the following year. The changes that were subsequently made to 
the syllabus due to this unexpected lack of interest from the students are 
detailed in Chapter 6. 

The survey consequently gave a list of seven possible reasons why the 
students had not chosen the modified syllabus and an “other” at the end for 
any reasons that had not been covered, and the students were requested to 
choose the reason that best conveyed why they had not chosen to follow the 
new syllabus (see Table 5.10 below for the list of responses). There was also a 
space for students to write an optional follow-up comment. The responses 
were chosen based on my experience with university students so responses 1- 
3 covered typical scenarios that had arisen in past courses and experiences 
with Economics students. Question 4 was included because one of the main 
complaints from students from my university is that they believe there is too 
high a workload. As it has been suggested that university students’ percep- 
tions about workload should be considered more often (Kember, 2004), this 
response was included for that reason. Questions 5 and 6 were included to 
understand how effectively the information about the Negotiated Portfolio 
had been communicated while Question 7 was included because I had re- 
ceived feedback from previous years’ students about how much they enjoyed 
reading Freakonomics. 
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The analysis of this survey was relatively uncomplicated as there were only 
eight responses possible so a simple count was made of the responses and a 
tally was created as can be seen in Table 5.10 below (n=63): 


Table 5.10 — Responses from end-of-course survey 


Single Multiple ! Total 


Response i 
response response i responses 

I don’t know what the Negotiated 1 0 l 1 
Portfolio is i 

I didn’t think it would be interesting 2 0 i a 
I forgot about it 3 1 i 4 
I thought it would be too much work 9 12 i 21 
I didn’t understand what it was so 3 2 ; 5 
preferred to do the Freakonomics Portfolio 

I don’t think there is enough support for 3 6 9 
the Negotiated Portfolio 

I like reading Freakonomics and wanted 21 13 | 34 
to work on that i 

Other (please specify): 2 4 6 
Totals 44 19 | 81 


As can be seen from the results above, 63 students took the survey but 81 re- 
sponses were generated. Because the survey was a simple pen-and-paper sur- 
vey that did not envisage one response only, some students put multiple re- 
sponses. Consequently, reading vertically, the “Single response” column lists 
the total number of students who only used one response (44) and a further 
19 completed the survey with more than one response. Reading horizontally, 
the total number of students who chose each response is shown in the final 
column. Therefore, a basic analysis that counted the responses was 
undertaken for questions 1-7. Question 8 “other” was answered by six 
students and the responses were coded using descriptive coding where 
descriptive labels were affixed to the responses to summarise the coded data. 
These codes were grouped into the categories that emerged, which were: 

- inability to find an interesting topic 

- lack of time 

- already read Freakonomics 
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One of these six responses did not seem to indicate that the writer had under- 
stood the question, and so was discarded. 

Question 2 in the survey asked “Please provide any other details you may 
have here” and only two people provided comments. One comment seemed 
to be answering the “other” question as the student had written: 


I thought it would be interesting to analyse, with an economic perspective, a book 
of my choice ... However I had no idea on how to elaborate an analysis of such a 
book. 


As this response seemed to be suggesting that the student had not been unable 
to find an interesting topic, it was coded with the other two responses for 
question 8. The other comment referred to the theme covered by response 7 as 
the student wrote: “I think it would be hard to find a comparable reading to 
Freakonomics”. 

The analysis of the first end-of-course survey therefore produced the re- 
sults that the students predominantly felt that reading Freakonomics, the set 
book that was the basis of the Portfolio that was the alternative to the Negoti- 
ated Portfolio, was more interesting than negotiating their own content for the 
Portfolio. Further they expressed the opinion that negotiating the Portfolio 
would be more work than doing the one that had been devised by me on 
Freakonomics. As such, therefore, it seemed clear that if I wanted more students 
to participate in the negotiated Portfolio, I would have to make the decision 
about the source material, whether that was Freakonomics or something else. 


5.6.1.2 The second end-of-course survey 


The second end-of-course survey, which was conducted in AR Cycle 3, was 
more detailed in its design than the first survey as it aimed to gather data in 
order to evaluate the redesigned syllabus that was used in this third and final 
AR cycle of the study. This survey used a mixture of closed yes/no questions 
about the course and then asked follow-up open questions to explain why the 
respondents had chosen yes or no. As discussed in Section 4.8.3.2, the purpose 
of using a more qualitative approach to data collection in this particular eval- 
uation stage was to generate findings that would be useful for the subsequent 
course the following year. Moreover, given the learner-centred approach to 
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the syllabus and due to my beliefs that the students should have a voice in the 
evaluation process, a qualitative approach that would provide a more detailed 
analysis was preferred. 

The second end-of-course survey was administered twice: once in Febru- 
ary 2016 with the PPE students and again in July 2016 with the E&M students. 
Both surveys were conducted on paper at the beginning of the students’ writ- 
ten examination. The surveys were used as an evaluation tool to assess how 
effective the Negotiated Portfolio in AR Cycle 3 had been, which had been 
based on Freakonomics. Some of the questions in the survey, however, were 
also designed to investigate various practical aspects of the course. As there 
had been a problem with irregular attendance in the PPE course, one question 
required students to state how much they had attended and the E&M end-of- 
course survey included a question relating to the introduction of extra credit 
exercises. As the analysis was extremely straightforward and only required a 
tally, the findings from these questions will be discussed in Chapter 7. The 
question that required more analysis, however, was Question 2, and some 
basic descriptive statistics concerning this question are presented in the table 
below: 


Table 5.11 — Responses to question 2 in the second end-of-course survey (* one student did not 
answer the question) 


Question PPE students n=15 E&M students n=90* 
Do you think it’s a good idea Yes No Yes No 

for you to be able to negotiate 15 0 86 3 

the contents of (some of) your 

course? 


This question was then followed with a request to explain why they thought 
it was a good idea (or not) to negotiate some of the contents of their course 
and was formulated as “If you have answered yes, could you briefly explain 
why you think it is a good idea?” As detailed in Section 4.8.3.2, the responses 
were coded inductively using content analysis, which “takes a volume of qual- 
itative material and attempts to identify core consistencies and meanings” 
(Patton, 2002, p. 453). The same sequence that was used for the coding of the 


interviews was also used in this case, in other words, first open coding, then 
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axial coding and then selective coding. As all the responses were handwritten, 
NVivo was not used to undertake this coding, and it was done manually. 

In order to prepare the data for analysis, the 105 surveys were numbered. 
Then during the open coding phase, the responses were read and recurring 
words and topics were highlighted and descriptive labels were developed in- 
ductively with the aim of “discovering patterns, themes, and categories” (Pat- 
ton, 2002, p. 453). These labels referred to the reasons why students thought 
negotiating the Portfolio was a good idea and comprised the following: 

- chance to decide 

- choose topics 

- amount of work 

- interests respected 

- motivation issues 

- individual needs 

- lecturer is open to discussion 


- cooperation with lecturer 


Once this first reading had been completed, the responses were read again 
and a word frequency analysis was taken in an attempt to identify further 
categories. This was undertaken in a similar way to NVivo’s word frequency 
analysis, which had helped to identify recurrent themes in the interviews. Fur- 
ther, like in the analysis of the interview data, the words were counted by 
matching words that had the same stem (e.g. “interest” and “interesting” are 
counted as the same word) and words that had similar meanings within the 
context were aggregated. Table 5.12 below shows the frequency analysis of 
the words used to directly answer Question 2, which was “If you have an- 
swered yes, could you briefly explain why you think [negotiating the contents 
of (some of) your course] is a good idea?” Only nouns, verbs and adjectives 
that were used to give reasons why they thought negotiation was a good idea 
were counted, and not words that were either off-topic, or were expected, such 
as “students”. Further the second line shows an example where a variety of 


words to express the same idea, choosing content, were aggregated. 
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Table 5.12 — Word frequency analysis of survey 


Word Count 
Interest / (Most) interested in / (Most) interesting 30 
Focus on / Choose / Influence topics / Content / Chapters / Subjects 25 
Good 20 
Give / Express / Share our opinion 9 
Amount of work / Workload 9 
Important 8 
Participation / Participate / Feel part of 7 
Decide (together) 6 
Enjoy / Like /Care about 6 
Motivated / Motivation 6 
Useful 4 
Be (more) involved / Involve 3 
Great 3 
Improve / Take into account our skills 2 
Preferences 2 
“To meet students’ needs” 2 
Cooperation / collaboration 2 
Compromise 1 
Contribute 1 
(More) personal 1 
Democratic 1 
A right 1 


Once this coding had taken place, the axial coding phase occurred where the 
in vivo and descriptive coding that had been undertaken was sorted and or- 
ganised into categories. As the use of conceptual frameworks had helped to 
organise the themes that had emerged from the interview data, the same ap- 
proach was used to categorise the students’ responses from the survey. The 
conceptual framework that emerged in this phase can be seen in the figure 
below. 
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As can be seen, the superordinate category stated the answer to the question 
that had been asked. Some students had answered the question using the ex- 


7 


pression “a good idea” (20 examples in the word frequency analysis) but 
others had used the other adjectives great, important and useful. The reasons 
why students thought negotiation was a good idea were divided into two 
main categories, which were coded as “Students can express their opinions”, 
to reflect the more practical, yet relational aspect of the students’ responses 
while others concentrated on the more participatory aspects of negotiation 
and so was coded “Students feel part of the learning process”. 

The “Students can express their opinions” category was divided into 
what students could express their opinions about, which comprised two main 
elements — the amount of work to be undertaken and the contents of the 
course. As already mentioned, students had tended to complain about the 
amount of work expected for this language course, so the mentioning of this 
aspect nine times (see word frequency analysis in Table 5.12) is not surprising. 
However, many more of the students who stated that negotiation was good 
because they could express their opinions tended to refer to the contents of 
the course, as the question that they were answering had suggested. As can 
be seen in Table 5.12, this aspect was coded 25 times. To highlight the im- 
portance of this aspect, therefore, the “choice of content” category in the con- 
ceptual framework is significantly larger than the “amount of work” category. 
The “choice of content” category is further subdivided into three subcatego- 
ries that comprise the aspects that the students mentioned about being able to 
choose the content. As can be seen from the word frequency analysis, the ref- 
erence to content that was “interesting” or that students were “interested in” 
was coded 30 times. In other words, almost a third of the survey responses 
that agreed that being able to negotiate the contents of their course was “a 
good idea” felt it was a good idea because they could negotiate content that 
they personally found interesting. As this was by far the most frequent reason 
stated, its visualisation as the subcategory in the conceptual framework is 
larger than the other two reasons, which were coded as reflecting the “needs” 
of the students and their “preferences”. The “needs” subcategory is the only 
one that contains a further element, which is a reference to skills work, which 


was explicitly mentioned in two surveys (see Table 5.12). 
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The other main category that answered the question was coded as “Students 
feel part of the learning process” in an attempt to capture the more par- 
ticipatory aspects of why the students felt negotiation was a good idea. The 
students used various expressions to refer to this category and the conceptual 
framework notes the words used. One word was “participate” and other 
forms of this lexeme, as in the following examples. “Because [negotiation] en- 
hances ... participation of the students” and “A person should always be able 
to participate in a discussion and negotiate for the desired aims”. 

However, the students also used other words to refer to the participatory 
aspects of negotiation, such as in the following examples where the word “in- 
volved” was used: “I believe [negotiation] makes students more involved” 
and “[negotiation] is a good way to involve the students and to let them ex- 
press their opinions”. 

Consequently, when respondents explained why they felt negotiation 
was beneficial, the participatory aspects, although less numerous than the 
practical aspects, were still frequent enough to merit their own category. 

Many students did not go beyond expressing anything further than the 
purely participatory aspect of negotiation, which was coded as “students feel 
part of the learning process”, but some did elaborate on the theme. Therefore, 
the two subcategories of “Joint decisions” and “Democracy” were integrated 
to reflect these aspects. One example that was coded as “joint decisions” was 
the following statement: 


students and professors often have different expectations of the portfolio, so they 


can explain them to each other, find a compromise 


Other examples that were coded in this subcategory included references to 
“cooperation” and “collaboration”. An example that was coded in the 
“Democracy” category is the statement that appears in the conceptual 
framework in Figure 5.15, where one student declared: “We have a right to 
give our opinion”. 

Given the fact that many of the categories that were generated in the cod- 
ing of the survey responses tended to intermingle, these relationships are 
shown in the conceptual framework with the connecting lines, the solid lines 
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show the primary connectors in the categories, whereas the dotted lines de- 
monstrate secondary connectors, which always referred to a result. To exem- 
plify this, the statement below was coded as “Joint decisions” but the result is 
“Choice of content”: 


I think it is a good idea because the professor can understand the interests of the 


students and take them into account during the preparation of the lectures 


The final category visualised in the conceptual framework is “Motivation”, 
which although appearing explicitly six times, always appeared as a 
consequence of one of the other categories. In the following statement, for ex- 
ample, motivation comes from the interest: “if students find [the course] in- 
teresting, they will have more motivation to do it” whereas another referred 
to the motivation coming from the ability to choose: “if we can negotiate the 
contents we can choose topics which we like and we are more motivated”. 
The analysis of this second end-of-course survey proved valuable in the 
evaluation of the implementation of the blended syllabus in AR Cycle 3, and 
contributed significantly to exploring the fifth research question of this study, 
regarding the evaluation methods needed to ascertain the effectiveness of this 


proposed syllabus. 


5.6.2 Analysing the data in AR Cycles 2 and 3 - Analysis of 
classroom-based data 


There were two other data collection stages in this study, the collection of data 
from the in-class worksheets used during the Negotiated Portfolio classes in 
AR Cycle 3 (see Section 4.7.1.3) and field notes taken during these classes (see 
Section 4.7.1.4). As previously discussed, the worksheet had two main aims, 
the first of which was to act as a framework for the students’ interactions 
during the discussion phase of the negotiated class so that the negotiation to- 
gether would be more focused and efficient. For the purposes of the study and 
the syllabus itself, the second aim of the worksheet was to provide evidence 
of what had been discussed and negotiated in class, given the importance of 
record keeping when implementing a negotiated syllabus (Clarke, 1991; 
Hedge, 2000). Therefore, requesting the students to provide me with their 
worksheets acted as a means to corroborate the notes that I had taken in class. 
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As the students were asked to provide me with their worksheets on a volun- 
tary basis, a total of 11 were collected from the PPE students in October 2015 
and 27 from the E&M students in March 2016 in AR Cycle 3. 

The field notes were taken twice, once in the Negotiated Portfolio class in 
AR Cycle 2 and again in the Negotiated Portfolio class in AR Cycle 3. The field 
notes taken in the classes aimed to describe what was being observed in order 
to act as further data to support the outcomes of the negotiated class, which 
were the contents and procedures concerning the Portfolio that were agreed 
upon with the students. 


5.6.2.1 In-class worksheets 


In order to fulfil the dual purpose of the worksheet, the questions in the 
worksheet were divided into three sections, entitled “A — Yourself”, “B — 
Freakonomics” and “C — The Portfolio”. Section A asked the students to reflect 
on their strengths and weaknesses in English. This question had two aims: to 
provide further data to compare with the data analysed from the question- 
naires and from the semi-structured interviews with the lecturers, but mainly 
to encourage the students to reflect on their own individual language needs. 
The analysis of this section involved reading the responses given by the stu- 
dents and comparing to what extent they corresponded to the responses from 
the questionnaires. The analysis comprised coding of the responses and given 
that the vast majority of the responses were presented as a list of mainly skills, 
the analysis was a relatively straightforward matter where tallies were col- 
lected for each skill or language aspect. Table 5.13 below presents the main 
strengths and weaknesses listed by the students in descending order, showing 
the top six mentions in each category (n=38). 


Table 5.13 — Students’ self-reported strengths and weaknesses taken from in-class worksheets 


Strengths Count Weaknesses Count 
Listening 27 Speaking 20 
Reading 27 Vocabulary 19 
Grammar 11 Grammar 17 
Writing 7 Writing 12 
Speaking 7 Pronunciation 6 
Vocabulary 5 Spelling / Presentations / Phrasal verbs 2 
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As can be seen from the analysis, the students who contributed their work- 
sheets felt that their main strengths were the receptive skills, as well as gram- 
mar, and only a few felt that their strengths were writing and speaking. 
Although most did not elaborate on their strengths and just listed them, some 
did provide reasons for them, such as one who stated that listening was a 
strength because “I listen to a lot of TV series and YouTube videos”. Others 
gave further information, such as “listening — I consider myself good at un- 
derstanding different kinds of pronunciation and generally good at extracting 
the key points from a general speech”. As far as the weaknesses were con- 
cerned, the main weakness was speaking, followed by vocabulary and gram- 
mar. Again, most just listed their weaknesses but some did provide further 
information, including reasons for their weaknesses. As far as speaking is con- 
cerned, two explicitly mentioned the fact that they felt anxious when having 
to speak English, and another wrote “I feel not comfortable in speaking even 
if probably I would be able to do it”. One even expressed a desire to have more 
speaking practice when they stated “Speaking. Although we do study in a 
trilingual university, I think there should be more opportunities to practice 
our speaking capacity”. Vocabulary was listed as the second most common 
weakness, but it was either almost always qualified with the adjectives “for- 
mal”, “advanced” or “specific”, with one student referring to “jargon”, or was 
described as “poor”, or was referred to in the negative with the adjective 
“wide”. 

Section B, however, focused on the content of the book and invited the 
students to discuss the more relevant chapters for their interests and needs. 
This section therefore aimed to provide them with the opportunity to negoti- 
ate the content of their Portfolio. Once the class had finished, I read this section 
of the worksheet and then compared the responses with the notes I had taken 
during the subsequent phase of the class when the whole group discussion 
was underway and the Portfolio was being negotiated. I made this compari- 
son to ensure that the notes I had taken reflected what had been discussed in 
the whole group negotiation as well as to ensure that as many students’ opin- 
ions were taken into consideration. During the classes when this negotiation 
occurred, the number of students present ranged from 15 to 62, and so not all 
students had the opportunity to give their own personal contribution in the 
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group negotiation phase, especially in the larger groups. However, given that 
the whole-class group negotiation took place after the discussion in smaller 
groups, every student present in the classes had had opportunities to engage 
in speaking practice. 

Section B had focused on the content to be negotiated, while Section C 
focused on the procedures to be negotiated, thus giving students the oppor- 
tunity to further identify needs, prioritise their aims, and choose what type of 
writing tasks they felt they wanted to do and how much they wanted to pro- 
duce for the Portfolio. The responses were again read and compared with the 
notes that I had taken during the negotiation phase as described previously. 


5.6.2.2 Field notes 


Field notes were taken in the negotiated class twice — once in the PPE course 
in October 2015 and again in the E&M course in March 2016. The field notes 
were taken while in class on both occasions and for the PPE course, they were 
written up on the computer a few days after the class. The field notes taken 
during the class tended to record a selection of the actions undertaken in the 
classes and some of the words uttered and as such were predominantly 
descriptive. Moreover, as they were written “in the heat of the moment” 
(Lankshear & Knobel, 2004, p. 229), they tended to be characterised by abbre- 
viations and other shorthand conventions, which is a typical feature of field 
notes taken in this way (Lankshear & Knobel, 2004; Miles et al., 2014). How- 
ever, K. Richards asserts that “fieldnotes can take different forms” (2003, 
p. 137) and can include more than just descriptions of what is being observed. 
I endeavoured to be non-judgmental, as recommended by Lankshear & 
Knobel (2004), and to concentrate on descriptions of the actions observed. 
However, the field notes taken also refer to “relational issues” (K. Richards, 
2003, p. 137) where my personal reflections as the teacher/researcher were 
noted. Given that “it is almost impossible to attribute states of being to others 
during observations where we only have external indictors to go on” 
(Lankshear & Knobel, 2004, p. 231), these reflections were expressed in a 
cautious manner. Examples of these from the second set of field notes taken 
in the E&M class included the comment “everyone seems fine” when referring 


to the students engaged in the preliminary discussion phase in small groups 
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after initial queries had been addressed. Another example of a personal 
reflection, although less cautiously worded, was this note referring to the 
attendance of students in Group B of the E&M course: “surprised there aren’t 
more present — obviously some aren’t more interested in active democracy”, 
demonstrating my surprise and frustration that fewer students than expected 
were in attendance that class. 

The field notes that had been taken in the PPE class were written up some 

days later, a procedure that is recommended by many (Lankshear & Knobel, 
2004; Gibbs, 2007; Miles et al., 2014). Writing up field notes is recommended 
because the process of writing helps to remember more details about the ob- 
served situation after the event: “raw field notes, when reviewed, stimulate 
the field-worker to remember things that happened at that time that are not 
in the notes” (Miles et al., 2014, p. 71). Moreover, similarly to the process of 
transcribing interview data, the “process of writing up is actually the first step 
in your qualitative analysis” (Gibbs, 2007, p. 27) as it allows the researcher to 
start reflecting on the data collected. Writing up field notes tends to involve 
recording what happened during the observed situation as well as recording 
the reflections of the observer. It has been suggested that these two aspects 
should be kept separate (K. Richards, 2003; Gibbs, 2007) as the field notes 
taken while observing can be regarded as primary data, whereas their writing 
up includes subsequent reflection and analysis of the observed situation. 
However, taking field notes is not “value-free” (Gibbs, 2007, p. 28) given that 
they will always reflect the human bias of the researcher, so the written-up 
field notes in this study incorporated both the descriptive elements from the 
hand-written notes, plus a follow-up reflection and analysis of the class. 
The analysis of field notes can be undertaken in various ways, including 
following a coding procedure that is often similar to the one detailed in this 
volume for the semi-structured interviews and students’ end-of-course sur- 
vey. However, given the fact that the field notes were primarily taken as a 
means to act as a memory aid for the design and then the actual implementa- 
tion of the Portfolio, and the value as a means of data collection for this study 
was secondary, they were only read through and stored. Discussion relating 
to the findings that emerged from these field notes is in Chapter 7 when refer- 
ring to the design of the modified syllabus. 
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5.6.3 Analysing the data in AR Cycles 2 and 3 — 
Comparison of the questionnaires 


The questionnaire that was administered to the students at the beginning of 
AR Cycle 1 was done in order to provide input for the needs assessment and 
analysis stage of this study. This questionnaire was then administered twice 
more at the same time of the year (May), in an attempt to obtain a longitudinal 
view of the students’ needs and English-language proficiency and difficulties. 
The data from the second and third questionnaires were analysed following 
the same sequence as the first questionnaire and as detailed in Section 5.4, but 
further analyses were run to compare the data between the three question- 


naires to understand to what extent the responses compared or were dissimilar. 


6. Findings From the Data Analysis 
in AR Cycle 1 and Discussion 


6.1 Findings From Data Analysis in AR Cycle 1 


The data collection and analysis that were detailed at length in Chapters 4 and 
5 occurred at various intervals throughout this three-year action research 
study. During AR Cycle 1, once the context had been problematised, a plan 
had been devised and the target populations identified, the collection and 
analysis of the quantitative data from the questionnaires to students and the 
qualitative data from the semi-structured interviews with the EMI lecturers 
took place. This collection and analysis were part of the target situation anal- 
ysis so as to better understand how English was used in the Faculty and to 
investigate the language problems that students encountered. As detailed in 
Section 5.2, following a convergent parallel mixed methods design, the two 
datasets were analysed separately and the results were merged and then tri- 
angulated to identify any similarities and differences between them and as 
such provide answers to the first two research questions. 

The first of the two datasets that were analysed was the questionnaire 
responses from the students. These responses had been transferred into an 
Excel file and were then transferred into SPSS for analysis. The other dataset 
analysed was the conceptual frameworks that emerged from the coding of the 
semi-structured interviews with the EMI lectures. Once these two datasets 
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had been analysed separately, the results were merged. Merging data in the 
convergent parallel mixed methods design, as discussed in Section 5.6, can be 
undertaken in different ways, and for this study, a side-by-side visualisation 
was used in the data analysis framework (Figure 5.2). However, as explained 
in Section 5.6, because the two datasets were very different in size and their 
analysis was visualised in very distinct and differing ways, the actual integra- 
tion of the data in this study will be presented as a discussion, “a popular way 
to represent integration” (Creswell, 2015, p. 84) in a mixed methods study, 
especially one with a convergent design, as is the case in this study. This dis- 
cussion will be presented integrated with the findings that are examined in 
the current and following sections. 

Before discussing the results from the two datasets, however, it is useful 
to outline the context that is being studied, and particularly how the classes 
using EMI are organised and delivered at the Faculty of Economics and Man- 
agement. The University is quite small in comparison to others in Italy. In Ta- 
ble 6.1 on the following page, data concerning the number of active student 
enrolments at the university as a whole, and the Faculty itself in the three ac- 
ademic years that were covered by the action research cycles are presented as 
well as data referring to the most recent full academic year (2021/2022)! 


Table 6.1 — Enrolment numbers for the University and Faculty in the period covered by the study 
and in the last full academic year 


2013-2014 2014-2015 2015-2016 2021-2022 


(AR 1) (AR 2) (AR 3) 
Faculty of 
Economics and 909 1028 1095 1164 
Management 
University 3615 3506 3392 4488 
1 Data retrieved from the Free University of Bozen-Bolzano — 


https://www.unibz.it/it/home/profile/ on 10 October 2022 
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As can be seen from the table, in the academic year that corresponded with 
this study’s AR Cycle 1, the Faculty had a total of just over 900 students en- 
rolled in its various degree programmes, which included the two undergrad- 
uate programmes that are the focus of this study, as well as on the other un- 
dergraduate degree course that takes place on a different campus, and the 
Faculty’s master programmes. The total numbers enrolled in E&M and PPE 
for the three academic years when this study took place are no longer able to 
be extrapolated as the university’s data system cannot filter and retrieve such 
data from previous academic years. However, for reference purposes, in the 
last full academic year (2021/2022) there were 413 students enrolled for E&M 
and 185 in PPE. 

Moreover, during the interviews with the EMI lecturers, it became clear 
that the classes using EMI tended to be lectures with relatively large student 
numbers, sometimes reaching 150 students to a class. This aspect was coded 
as described in Section 5.5.3.2 and was included in the “Problems encountered 
in the target situation” conceptual framework that was presented in Figure 
5.10, specifically in the “Type of class” subcategory. The style of teaching used 
in these EMI classes was variously described by the lecturers as “old-style”, 
“formal”, “frontal”, “classical” and “face to face”. This style of teaching seems 
to be relatively typical in Italian university contexts as has been demonstrated 
in the results from a national survey of English-medium instruction in Italian 
universities, which comprised a questionnaire survey sent to all the 76 univer- 
sities in Italy (Costa & Coleman, 2013). In this survey, over 70% of the univer- 
sities that responded confirmed that formal lectures were used more than 70% 
of the time when delivering classes taught in English, as opposed to seminars 
which would typically have fewer students and would almost undoubtedly 
provide more opportunities for interaction between the students and lecturer. 
Therefore, the data collected in this other study about how lectures are con- 
ducted for EMI classes, “testify to a very traditional type of teaching style” 
(Costa & Coleman, 2013, p. 14) and as such, largely correspond with the find- 
ings that characterise the context under study for this volume. 

The following sections will now examine the main findings that emerged 
during the reflection stage of AR Cycle 1 when the quantitative and qualitative 


data were merged. There will be a discussion of the quantitative results from 
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the questionnaires that directly relate to the first and second research 
questions, and these will be compared with the findings from the interviews 
to identify how far the findings converge. Once these findings have been pre- 
sented and discussed, I will describe how they informed the design of the 
syllabus that was implemented in the action stage of AR Cycle 2 (see 
Figure 6.1). 


6.1.1 Research Question 1: Discussion of quantitative results with 
comparison of findings from interviews 


In order to answer the first research question, and referring to the question- 
naires completed by the students, the responses to Questions 8-11 are im- 
portant as these questions requested students to provide an answer to “What 
do you have to do in English when at the unibz?” for reading, listening, writ- 
ing and speaking. Each question provided a list of activities and respondents 
were able to choose as many of the items as were relevant. Table 6.2 below 
displays the results for question 8, referring to reading, showing the relative 
frequency of each choice based on the total number of respondents. 


Table 6.2 — Answers to the question “What do you have to do in English when at the unibz?” — 
reading, as percentages of total number of respondents 


Reading activities undertaken at the university % 


Read study material from professors 96.15 
(e.g. slides, assignments, handouts) 


Read text books 95.60 
Read emails 95.60 
Read journal articles 58.79 
Read financial documents 48.35 
Read research reports 47.80 
Read business reports 37.91 
Read contracts and agreements 28.02 
Read technical manuals 14.84 
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As can be seen, the vast majority of respondents (over 95%) selected the three 
most frequent choices which referred to reading material prepared by the 
lecturers, reading textbooks and reading emails. Reading study materials 
(assigned by lecturers) implies that students have to read both during class 
time and outside class time and although the questionnaire made no provision 
to discover when this reading occurred, the lecturers provided further infor- 
mation regarding this point. Almost all of them mentioned that reading was 
envisaged during the class, including reading slides and handouts. Only 
Rodion, the Mathematics lecturer and Ottavio, the Information Systems lec- 
turer did not specifically refer to reading texts in class. This finding corre- 
sponds with results from the survey of English-medium instruction in Italian 
universities, which reported that PowerPoint presentations were the most 
popular teaching material used by EMI lecturers and that this tendency was 
growing (Costa & Coleman, 2013). 

Most of the lecturers also referred to reading outside of class, including 
reading textbooks, handouts and in the case of Philosophy, parts of longer 
texts. The students concurred that reading textbooks was a frequent activity 
as it was in fact the joint second most frequent reading activity that they had 
selected. “Reading to learn” (Grabe, 2009, p. 9) is an activity that is often car- 
ried out by students in academic contexts typically using textbooks or material 
provided by the lecturers, and it has even been suggested that “the greatest 
need of students is the ability to read textbooks” (Jordan, 1997, p. 50). This 
finding was also consistent with the national survey, which found that the use 
of textbooks in EMI courses was almost as frequent as the use of presentations 
(Costa & Coleman, 2013). Consequently, the findings from the analysis of the 
two datasets regarding what texts were commonly used in the context under 
investigation in this study tend to correspond consistently with findings from 
other Italian universities and from international studies. Reading emails was 
also a frequent activity according to most students and was in joint second 
place with reading textbooks as the most frequent reading activity. This result 
is unsurprising given the ubiquity of email use in everyday life as well as at 
the University. 

It is beyond the scope of this chapter to present the full results from the 
other questions seeking information on the other skills, but Table 6.3 below 
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summarises the most frequent responses for each skill, with the percentage 
referring to how many respondents selected each one: 


Table 6.3 — “What do you have to do in English when at the unibz?” — the other skills as 
percentages of total number of respondents 


Listening activities undertaken at the university % 
1 Listen to lectures 98.90 
2 Listen to presentations given by other students 79.56 
Writing activities undertaken at the university % 
1 Write exam answers 94.97 
2 Write emails 93.85 
3= Write essays 74.86 
3= Write lecture notes 74.86 
Speaking activities undertaken at the university % 
1 Engage in interaction with professors (e.g. in class and/or 86.21 


in office hours) 
2 Make formal presentations in class 81.61 


3 Engage in interaction with other students 69.54 


The two most frequent listening activities that students stated they engaged 
in were listening to lectures and listening to presentations given by other stu- 
dents, presumably in those same lectures since the teaching model at the Fac- 
ulty only provides one type of class delivery. Listening to lectures is unsur- 
prisingly in top position given that the lectures were all delivered in English. 
Perhaps what is unexpected is that it does not reach 100%, although it is likely 
that this is due to some questionnaire respondents being non-attending stu- 
dents, either because they were “fuori corso” (see Section 5.4 for an explana- 
tion) and so were no longer attending lectures, or simply because attendance 
is not compulsory at the Faculty. The second most common listening activity 
was listening to presentations given by other students selected by nearly 80% 
of the respondents. As the undergraduate programmes that were the focus of 
this study progress into the second and third year, the number of attendees in 
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many classes generally decreases due to the students being able to choose spe- 
cific courses outside the core courses that comprise the two degree pro- 
grammes, which are still delivered as large lectures. Therefore, it is not sur- 
prising that of all the respondents who stated they listened to presentations 
given by other students, over 60% (n=96) were from the second year and 
above, indicating that this practice is relatively common in second and third- 
year classes. 

As far as writing is concerned, the questionnaire contained many more 
options for writing activities than for the other skills, given the prevalence of 
writing at the Faculty, and the most frequent was writing exam answers, 
which was selected by almost 95% of respondents. As all the exams at the Fac- 
ulty are required to be written exams, which has been noted as being typical 
in Italian universities (Costa & Coleman, 2013), this result was not unexpected. 
Emailing was the second most frequent activity, which mirrored how fre- 
quently emails appeared in the results regarding reading, and thus confirmed 
the prevalence of email correspondence at the university. The third most fre- 
quent writing activity was jointly writing essays and writing lecture notes, 
selected by three-quarters of the respondents. Writing notes in lectures is a 
common and critical activity at undergraduate level (Badger et al., 2001; Boyle 
& Forchell, 2014) and its frequency here was unsurprising. Writing essays, 
however, was an unanticipated addition to the list given the fact that many 
courses that were being taught in English when the data were collected for 
this study were subjects connected to mathematics, finance and computing 
skills and as such would not necessarily require essay-writing skills. Indeed, 
comparing this finding from the questionnaires with the interview data shows 
that very few EMI lecturers made any reference to essay writing. Two who 
did mention essay writing were the two PPE lecturers: Oscar, the lecturer of 
the Principles of Philosophy course used a “take-home essay” as a means of 
assessment, and Benno, the lecturer for Political Science 1, stated that his stu- 
dents had to “write like a small essay in English” for his exam. These two 
subjects, however, were the two most “language heavy” of the subjects taught 
in English at the time of the study and in any case the PPE students only com- 
prised just over a fifth (21.85%) of the respondents in the questionnaire con- 
ducted in AR Cycle 1. Dario also mentioned that in his exam the question “[is] 
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always a short essay”. However, Dario specifies later that he taught third-year 
students on the Economics and Management course, which would only ac- 
count for approximately a third of that particular student cohort over all. Con- 
sequently, this incongruity in the findings from the two datasets could indi- 
cate a limitation of the questionnaire in this study and questionnaires in gen- 
eral, which often “provide a rather ‘thin’ description of the target phenomena” 
(Dérnyei, 2007, p. 115). It was noted in another study where results from ques- 
tionnaires diverged that respondents can “rush through the questionnaire giv- 
ing token responses” (Huang, 2010, p. 535) and this could therefore lead to the 
responses being inaccurate. In the case of the study that is the focus of this 
volume, it is likely that either the students misinterpreted what “essay writ- 
ing” actually means, or this response is an example of a “social desirability 
bias” (Dérnyei, 2007, p. 54) as discussed in Section 4.6. 

The final category concerned speaking and the most frequent speaking 
activity, chosen by 86% of the students, was engaging in interaction with lec- 
turers either in class or in office hours. The interview data tended to confirm 
this finding as spoken interaction taking place between the lecturer and stu- 
dents in class was mentioned as an activity by all the lecturers, and three also 
mentioned using English when speaking to students during their office hours 
(Dario, Ivan and Claire). Although the findings from both datasets revealed 
that speaking was a frequent activity undertaken by students with their lec- 
turers, there was no indication as to how long these interactions lasted nor 
whether they were useful for the development of the students’ speaking skills. 
In Costa & Coleman’s national survey of EMI courses, however, not only did 
they use a questionnaire to gather data but they also used case studies where 
a number of EMI classes were observed. They found that in these observed 
classes “there was little interaction and very few occasions for the students to 
speak” (2013, p. 15), despite the lecturers surveyed claiming that their classes 
developed speaking skills. This implies, therefore, that even if students and 
lecturers believe that a skill is practised frequently, the type of practice that 
Costa & Coleman witnessed in the classes they observed, which broadly cor- 
respond to the classes that typically take place in my context regarding the 
teaching style and number of students, was not necessarily useful for skill de- 
velopment. In order to develop undergraduate students’ speaking skills, 
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therefore, just attending classes where English is a medium of instruction does 
not seem to be sufficient, and an approach would be needed that could pro- 
vide not only a greater number of opportunities for speaking but that these 
opportunities would also need to provide more prolonged interaction. This 
consideration was therefore fundamental when designing the ESAP syllabus. 

The second most frequent speaking activity chosen by the students was 
making presentations in class. However, the data from the interviews revealed 
that only Riccardo stated that he used this type of activity in their classes: 
“usually at the end when they have solved the exercise they have to present 
the solution” whereas Oscar stated categorically “there are no presentations 
or anything else” and Benno stated that he had ceased using presentations in 


class: 


I used, you know, like midterm and in-class presentations for all the students in 
[the] first year I was here, but I... it’s difficult for me to explain why, but I realized 


this was not helpful 


Therefore, the data from the lecturers tended to refute the students’ assertions 
that they were required to give presentations in class. 

The third most frequent speaking activity mentioned by nearly 70% of the 
students was speaking to other students. However, the questionnaire did not 
make provision to identify where this interaction with other students would 
take place and whether the interaction was related to their academic tasks or 
was of a more social nature. 

From the findings generated by the responses to questions 8-11 in the 
questionnaires, therefore, the activities that students engaged in most fre- 
quently concerning the receptive skills were reading study material, textbooks 
and emails, and listening to their lecturers and other students giving presen- 
tations in class. As far as the productive skills were concerned, the most fre- 
quent writing activities were writing exam answers and emails and the most 
frequent speaking activities were interacting with academic staff and other 
students, and making presentations. 

Not only did the questionnaire seek to discover effectively what skills 
were being used in the target situation, it also sought to investigate how often 
the skills were used. This was done in order to provide more data for research 
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question 1, and to investigate whether there was any relationship between the 
frequency of practice of a skill and students’ self-reported proficiency in each 
skill to provide data for research question 2. The data were simply generated 
as frequencies from the first questionnaire results in SPSS. 

As can be seen from the charts in Table 6.4 below, the most frequent skill 
practised was reading, which was practised by more than half of the respond- 
ents every day. Listening was practised by almost half of respondents every 
day whereas the productive skills were practised less frequently although 
over three-quarters of the students identified that they spoke and wrote Eng- 
lish at least once a week for their studies. 


Table 6.4 — Frequency of skills practice from first questionnaire data 


Reading Number % Listening Number % 
Once a month 9 6.0 Once a month 5 3.3 
Once a week 12 7.9 Once a week 18 11.9 
2-3 times week 44 29.1 2-3 times week 58 38.4 
Every day 86 57.0 Every day 70 46.4 
Total 151 100.0 Total 151 100.0 
Writing Number % Speaking Number % 
Never 4 2.6 Never 9 6.0 
Once a month 28 18.5 Once a month 23 15.2 
Once a week 43 28.5 Once a week 40 26.5 
2-3 times week 48 31.8 2-3 times week 48 31.8 
Every day 28 18.5 Every day 31 20.5 
Total 151 100.0 Total 151 100.0 


Although the questions regarding frequency of skill use did not endeavour to 
investigate whether the skills practised were being used in the classroom, in 
the exam or for self-study, the data generated showed that the students en- 
gaged extremely frequently with the receptive skills and less frequently with 
the productive skills. As reading was the skill that was used the most often, it 
can be assumed that the sheer variety of reading texts that were identified by 
the students in question 8 of the questionnaire (see Table 6.2) contributed to 
the frequency of the reading done. Moreover, reading can be undertaken in 
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many different situations, from in the library to on the bus, and texts are now 
available in various forms, from the traditional textbook to digital versions 
that can be read on a phone, so it is unsurprising that reading is the skill with 
which students most frequently engaged. Moreover, a crucial aspect of read- 
ing in academic settings is that “when students read it is for a purpose” (Jordan, 
1997, p. 143 original italics). However, there is not only one, but numerous 
purposes for reading in academic settings, including reading for information, 
reading to learn, and reading to integrate information, amongst others (Grabe, 
2009, p. 8). Given the numerous purposes for reading in academic contexts, as 
well as the number of different texts to read and the relative ease to read in 
many contexts, it is understandable that reading was the skill that was 
undertaken the most frequently. 

The other receptive skill, listening, was also undertaken very frequently, 
with over 80% of the students stating they listened to English at least 2-3 times 
a week. This can certainly be linked to the fact that almost all of the students 
had identified that they listened to lectures in English, and these lectures 
where English was used as a medium of instruction must have been scheduled 
on a very regular basis due to the reported frequency of listening to English. 
Since the subjects taught by the lecturers interviewed for the study occurred 
in both PPE and E&M at the Faculty as well as across the three years of the 
programmes, these data seem to indicate therefore that most students were 
regularly attending classes where English was used. Consequently, listening 
to lectures in English was not only a prevalent activity, but it was evidently 
recurrent and regular. 

These data also showed that although there was a high frequency of prac- 
tice in the receptive skills, the productive skills were practised less often. This 
phenomenon can also be seen in the responses to the question “What do you 
have to do in English when at the unibz?” in Table 6.4, where the top speaking 
activity was “engaging in interaction with professors” but which was only 
chosen by 86% of the respondents, as opposed to the most frequent reading 
and listening activities that were selected by over 95% of respondents. Alt- 
hough the students were studying at a trilingual faculty, and it seems evident 
that lectures in English were held regularly, it is likely that there was not an 
EMI lecture every day of the week, given the number of other subjects that 
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were taught in Italian and German. Moreover, given the class sizes and affec- 
tive aspects related to the context, which will be discussed at length when 
considering the findings from the interviews, even if students had been regu- 
larly attending classes taught in English, they might not have had many, if 
any, opportunities to engage in spoken interaction with the lecturer. There- 
fore, if students chose not to attend office hours when offered in English, 
which would have provided them with an opportunity to interact at length in 
English with the lecturers, then it is not surprising that speaking was practised 
less often than the other skills. Similarly, as the main writing activity identi- 
fied was “writing exam answers”, exams were not held on a weekly basis and 
so this will also have had an impact on the frequency of practice. 

The questionnaire, as well as functioning as a means to gather data on the 
target situation, was also devised as a present-situation analysis to estimate 
the strengths and weaknesses in the language skills. Asking students what 
level of difficulty they felt they experienced in the four skills (question 20) 
would therefore provide further data to answer the first research question. 
Moreover, the questionnaire required the students to provide information 
about their perceived proficiency levels in the skills and what levels they con- 
sidered were needed to study at the Faculty at the time they were completing 
the questionnaire (questions 21—24). For these two aspects, as explained in sec- 
tion 5.4.4, descriptive statistics were generated using SPSS to create ordinal 
data to identify which skills were regarded as being the most problematic for 
the students. The responses to question 20 were coded from 1 to 4 to refer to 
the four levels of difficulty used: very difficult, quite difficult, quite easy and 
very easy. As can be seen from the chart reproduced from SPSS in Table 6.5 
on the next page, the skill that was evaluated as being the least difficult with 
a mean of 3.42 was reading, followed by listening with 3.14. Speaking was 
rated with a mean of 2.77, while the most difficult was writing with a mean of 
2.53. 
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Table 6.5 — Descriptive statistics of responses to question 20. Difficulty was rated as 1=very 
difficult, 2= difficult, 3=easy, 4=very easy. Note: time 1 refers to the first questionnaire of the three 
administered over the AR cycles 


Descriptive Statisticsa 


N Minimum Maximum Mean Std. Deviation 
DifficultyR 151 2.00 4.00 3.4172 56988 
DifficultyL 151 2.00 4.00 3.1391 62225 
DifficultyW 151 1.00 4.00 2.5364 .77266 
DifficultyS 151 1.00 4.00 2.7748 84988 
Valid N (listwise) 151 


a. Time = time 1 


Further analysis of Table 6.5 shows the standard deviation (SD) in the final 
column, which gives an indication of the spread of all the responses and as 
such measures the variance from the mean. The SD for reading is .569, which 
signifies that the majority of students’ responses were within .569 units above 
and below the mean of 3.42. This indicates the scores obtained from the sam- 
ple were relatively homogenous with regard to reading. In contrast, speaking 
had the highest SD with .849 from the mean of 2.77 signifying that there was 
greater variance in the responses and so the scores were more heterogeneous 
than for reading and indeed the other skills. Consequently, the SD is 
important in this case as it provides further information about the level of 
difficulty of the skills. Although writing had the lowest mean, which suggests 
that it is the skill that students find the most difficult, its SD was lower than 
that of speaking, which means that the scores were more tightly clustered 
around the mean. The mean for speaking was higher than that of writing, 
which suggests that speaking is less difficult for students, but its SD is higher. 
This, therefore, demonstrates that it is in fact speaking that these students feel 
is the most difficult skill. 

The findings from these data indicate that reading is regarded by students 
as the least difficult skill, which has been noted in the past (Jordan, 1997). 
However, a limitation of the questionnaire used in my study was that it only 
referred to the main skill and not all the subskills that are typically needed in 
academic reading such as skimming, scanning, inferences, distinguishing 
between facts and opinions, important and unimportant information and so 


on. Because of this, there is a possibility that the respondents overestimated 
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their ability in reading, which was also the case in a similar study where stu- 
dents were also asked to complete self-assessment surveys (Huang, 2010). 
There have also been concerns raised with using self-assessment data from 
learners in needs analysis since the data they provide may not always be reli- 
able (Auerbach, 1995; Long, 2005c; Huang, 2010), which is one of the reasons 
why this study used a mixed methods approach when collecting the data and 
approached two target populations. As far as the productive skills were con- 
cerned, however, similar to other studies that examined language difficulties 
faced by students on English-language programmes (Jordan, 1997; Huang, 
2010; Evans & Morrison, 2011a), writing was considered to be more difficult 
than reading and listening by the respondents in my study. However, one sig- 
nificant divergence in my study compared to these other studies is that speak- 
ing was recognised as being the most difficult skill overall (Prior, 2021). 

The responses to questions 21-24 were also analysed using descriptive 
statistics and the results for the four skills can be seen in Table 6.6 below. As 
noted, the six responses available were coded from 1 to 6, with 1 correspond- 
ing to the A1 level and 6 corresponding to the C2 level from the CEFR. More- 
over the first column shows R, L, W and S referring to reading, listening, writ- 
ing and speaking and each skill is divided between “I can” and “I have” to 


following the question format in the questionnaire. 


Table 6.6 — Descriptive statistics of responses to questions 21-24. Difficulty was rated as 1=very 
difficult, 2= difficult, 3=easy, 4=very easy. Note: time 1 refers to the first questionnaire of the three 
administered over the AR cycles 


Descriptive Statistics* 


N Minimum Maximum Mean Std. Deviation 
IcanR 151 3.00 6.00 4.8079 .77214 
ThavetoR 151 1.00 6.00 5.1391 99358 
IcanL 151 2.00 6.00 4.7417 85216 
ThavetoL 151 1.00 6.00 5.0530 1.00524 
IcanW 151 1.00 6.00 4.4636 85459 
ThavetoW 151 1.00 6.00 5.1192 96559 
IcanS 151 2.00 6.00 4.3907 96591 
ThavetoS 151 1.00 6.00 4.9669 1.04828 


Valid N (listwise) 151 
a. Time = time 1 
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Certain significant results are immediately apparent. The mean value for all 
the skills for the “I can” response is always lower than the mean for the “I 
have to” response, implying that the students generally felt that their level in 
all the skills was not as high as the level needed in those skills to study at the 
Faculty. Moreover, there is a clear distinction between the receptive skills and 
the productive skills, where the receptive skills are rated at a higher profi- 
ciency than the productive skills (see figures in bold in the table), with reading 
at 4.80 and listening at 4.74 but writing at 4.46 and speaking at 4.39. These 
results mirror the findings from the question discussed previously, where stu- 
dents stated that they generally had a lower level of difficulty with the recep- 
tive skills than the productive skills. 

Observing the standard deviation values that are presented in table 6.6, it 
can be seen that there is more variance in the “I have to” results than the “I 
can” results, perhaps suggesting that the students were somewhat unsure 
about what levels were required for their studies. The fact that the minimum 
value chosen for each skill in the “I have to” category was 1, which corre- 
sponded to the A1 level, would indicate that some had either underestimated 
the levels required, or had not necessarily understood or engaged with the 
questionnaire. Further analysis of the SD results shows that there is a differ- 
ence in the SD of the “I can” results, where the value for reading (.77) is sig- 
nificantly lower than for the other skills. Again it seems that the students rated 
themselves at a generally higher level in reading than any other skill, given 
the minimum value that was selected for reading was 3 (which corresponded 
to the B1 level) and “I can” values for all the other skills were at 2 (A2) or 1 
(A1). Moreover, comparing the SD values across the four skills shows that 
speaking showed the most significant variance (.96), indicating that, again, the 
respondents’ self-reported proficiency levels in speaking were much more 
heterogeneous than in the other skills. 

Furthermore, all the skills were rated at a mean of between 4.4 and 4.8 for 
the “I can” responses, which is above B2, which was coded 4, and approaching 
the C1 level, which was coded as 5. Consequently, it can be supposed from 
these results that the students believe they have, on average, a good B2 level 
in all four skills. The ESAP course that is the subject of this study is pitched at 
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the C1 level and takes place in the second year of the two undergraduate pro- 
grammes. In order to be able to enrol for undergraduate programmes at the 
Faculty of Economics at the time of the study, students would have had to 
have certified a B2 level in two of the three teaching languages of the Univer- 
sity (German, Italian and English). The third language had to be certified at 
the B1 level before students could enrol into the second year of their studies. 
However, 26% of the questionnaire respondents were first year students, so 
some of them would not have certified a B2 in English in order to enrol. Fur- 
ther only 81 students, which corresponded to just over half of the 151 students 
who responded to the questionnaire administered in AR Cycle 1, stated they 
had any international language certificate as can be seen in Table 6.7 below. 
Consequently, it seems questionable that the general level across the four 
skills would be at this high B2. 


Table 6.7 — Data from first questionnaire regarding international certificates 


Certificate n A B C B1 Pass (FCE) B2 Pass (CAE) Grade 


FCE 44 8 22 13 1 - 
CAE 22 2 6 11 - 3 
CPE 1 1 - - - - 
IELTS 11 
TOEFL cae 

81 
No 
certificate 70 _ 
Total 151 


In summary, the quantitative data gathered from the students in response to 
the first research question showed that as far as frequency was concerned, the 
skills most “needed” were the receptive skills, reading and listening, followed 
by the productive skills, writing and speaking. Over 90% of students stated 
they read or listened to English at least once a week, dropping to just under 
80% for writing and speaking. However, as mentioned, the questionnaire also 
sought to investigate the estimated strengthens and weaknesses of the stu- 
dents for each skill and as such functioned as a present-situation analysis of 
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their language needs. Regarding this aspect, the findings in fact showed the 
opposite trend, where students had greater difficulties with the productive 
skills than with the receptive skills and in particular, they reported that speak- 
ing was the skill that presented the greatest level of difficulty. 

Further data were collected from the questionnaires but for the purposes 
of providing information for the first research question of this study, the ques- 
tions and their findings that have been discussed in this section were the most 
relevant and useful. These findings, as discussed, were merged with data col- 
lected from the interviews as a means to achieve triangulation and identify 
the similarities and differences between the two datasets referring to the first 


research question. 


6.1.2 Research Question 1: 
Discussion of findings from qualitative analysis 


The interviews that were conducted in AR Cycle 1 in order to gather further 
information about the target situation took place with ten lecturers who were 
teaching subjects using EMI. The details concerning these lecturers were al- 
ready presented in Chapter 5, Section 5.5. 


6.1.2.1 Findings from the “Use of English” conceptual framework and 
implications 

The data collected and analysed from the interviews with the academic staff 
tended to intersect with much of the data analysed from the questionnaires 
concerning the skills needed and used in the subjects using EMI, although 
there were some important divergences as has already been specified. The 
“Use of English” conceptual framework depicted in the previous chapter pre- 
sented the skills that were mentioned by the academic staff as being used in 
the classroom and for the exam. In the classroom, reference to all four skills 
was coded and integrated into the “Language skills” subcategory. Reference 
to the skills used in the exam, however, was included in the “Language skills” 
subcategory emanating from the “For the exam” main subcategory, as can be 
seen from the conceptual framework reproduced below in Figure 6.2. Alt- 
hough all four skills were used in the classroom, the main skill mentioned for 


the exam was writing, with only one reference to reading. 
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The order that the skills are presented in the conceptual framework above 
shows the frequency of the references made, so speaking in the classroom was 
referred to more often than any of the other skills. As discussed in Section 
5.5.3.1, a word frequency analysis was undertaken using NVivo to establish 
this aspect, and Table 5.9 presented the frequency of the references to the skills 
and their subsequent coding into “Language skills” but also into “Academic 
skills” or “Study aids”. Below in Table 6.9, the same results have been aggre- 
gated to show more clearly how the four language skills were referred to. Only 
explicit references to the skills have been included in this aggregation so “ask 
questions”, which were the words that were most frequently coded in the 
“Language skills” category, was excluded. Many EMI lecturers stated that 
they asked questions in class, and the fact of asking questions would imply 
that students would have to answer. Therefore, this was coded as a “language 
skill” in the second axial coding phase as explained in Section 5.5.3.1. How- 
ever, it is not possible to extrapolate any further findings concerning which 
skills the students would have used to answer these questions, given they 
could answer using spoken or written language, thus the exclusion of this par- 


ticular value. 


Table 6.8 — Aggregated references to skills use in the classroom from the interview data 


Skill In vivo coding aggregated No. of occurrences Total occurrences 
Speaking “talking” 16 
“interact” 14 
“discuss” 11 
“speaking” 8 49 
Reading “reading” 17 17 
Writing “notes” 10 
“writing” 10 20 
Listening “listen” 8 8 
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From this analysis shown in Table 6.9, therefore, it can be seen that speaking 
was the skill used in the classroom that was referenced the most frequently 
with 49 occurrences in total, considering the synonyms and other related 
words for speaking. In fact, all of the lecturers, except for Rodion, the lecturer 
for Mathematics for Economists A & B, explicitly mentioned that the students 
were required to speak English in their classes. The next most frequent activity 
that was mentioned was “reading”, followed by “writing” with twenty occur- 
rences, and “listening”. As can be seen from the table, there were ten occur- 
rences of the word “writing” yet there were also ten occurrences of references 
to “notes”, which, in the conceptual framework, was coded separately as the 
academic skill “note-taking”. For this particular analysis in Table 6.9, how- 
ever, it is included in the “writing” category given that taking notes is a writ- 
ing activity. 

The conceptual framework in Figure 6.2 also depicts the skills used in the 
exams administered for the different courses taught in English and the main 
skill that was referred to was writing, which occurred 17 times. As mentioned 
previously, all the exams at the Faculty are administered as a written exam, 
and no oral exams are required apart from in the exams for the various lan- 
guage for specific purposes courses, which are not only offered in English but 
also in German and Italian. Therefore, the fact that writing was the main skill 
that was referred to being used in the exams is not surprising. However, some 
lecturers mentioned that the exams they administered did not even require 
much writing in English given that some of the courses taught in English at 
the time were mathematical or financial subjects. Ivan, the lecturer for Finan- 
cial Analysis stated that “the exam is on the laptops, they do the exercises 
which are very much numerically based so they don’t have to write lots” and 
Riccardo, who taught Financial Risk Management, stated “they don’t need 
English it’s just maths” when referring to the questions used in the exam. The 
other skill that was referred to for the exam was reading, but this was only 
mentioned by Ottavio when discussing the students’ problems in his exam. 
Consequently, the use of skills mentioned by the lecturers in the interviews 
tended to focus more on the multiple skills needed to be used in the classroom, 
rather than the skills needed in the exam. 
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However, one aspect that emerged clearly from the “Use of English” concep- 
tual framework concerning the type of writing used in the exams was the fact 
that when students were required to write for their exams, as opposed to using 
mathematical calculations or formulas, the question types mentioned by the 
lecturers often referred to specific types of writing that were “short”. There 
were three references to “short answers”, two to “short essay”, one to “short 
small questions essay-like” and one to “short paragraphs”. These findings can 
be compared with the thirteen genre families that Nesi & Gardner (2012) have 
identified in student writing in higher education, and the types of writing 
texts identified by the lecturers in my study seem to correspond almost exactly 


to the “explanation” and “exercise” genre families, where 


Explanations are set to develop students’ understanding of an object of study, and 
their ability to describe it and account for its significance, whereas Exercises are 
intended to give students practice in key skills and consolidate knowledge of key 
concepts. (Nesi & Gardner, 2012, p. 61) 


Not all the lecturers gave reasons why they asked students to produce these 
writing tasks in the exam, but some of the reasons included references that 
can be interpreted as being part of the “explanation” genre. Dario stated the 
exam questions he set required the students “to comment the result”, which 
seems to be referring to accounting for significance. Another reference to “ex- 
planations” was in this statement from Oscar, the Principles of Philosophy 
lecturer, who stated that in the exam he provided: 


questions ... to understand who really understood something ... and then I give a 
bit more complicated questions or a question which involves a step more than just 


remembering and having more or less understood. 


This statement appears to indicate that some of the questions he sets for his 
exam aim to develop understanding through an ability to provide descrip- 
tions. A reference to the “exercises” genre family, however, comes from Otta- 
vio, who describes how he required students to describe a database using a 
“schema” or framework, which could be completed with key words only and 
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did not need full sentences. However, he did also state that “any further ex- 
planation will help me understand what they are doing”, which seems to be 
clearly referring to consolidating key concepts. He also made a reference to 
the “explanations” genre when referring to a different exam he held: “I expect 
them to produce, let’s say, an answer of at least 5-10 lines, so rather short 
answer but in full English with a description”. As can be seen from these find- 
ings from the data, and as presented by Nesi & Gardner (2012, pp. 61-62), one 
of the key features of both “explanations” and “exercises” is that they tend to 
be shorter than the other genre families that were studied, such as essays, case 
studies or reports. Moreover, Nesi & Gardner (2012, p. 72) identified that 
when exercises are used in business courses, they tend to be used in account- 
ing and finance or statistics courses, as well as in mathematics and computer 
science, which largely corresponds with the courses that were taught by the 
lecturers interviewed for this study who made reference to this aspect. 

The other reference to writing for the exam came from Claire who referred 
to writing reports, when she stated, “In the first year they have to do three 
assignments which are basically technical not really a lot of writing to do but 
I also have them write a business report”. This report, although for the exam, 
was to be completed as an assignment and in fact she stated later that “they 
do it in groups”. Therefore, although this piece of writing was for the exam, it 
could be prepared by more than one student in their own time rather than in 
traditional exam conditions. 

The main findings from the analysis of this first conceptual framework 
concerning the language skills needed therefore showed that all the language 
skills were required in the classrooms that were studied, but only writing was 
required in the exams. Of the skills needed in the classroom, speaking was the 
most frequently mentioned, followed in order of frequency by reading, writ- 
ing and listening. 

Further findings from this first conceptual framework concerned the 
other characteristics that were coded in the “In the classroom” category, which 
were “Academic skills” and “Study aids”. As discussed in Section 5.5.3.1, 
these were coded as being distinct from the “Language skills” category, alt- 
hough clearly they are still connected to the language skills, thus the connect- 
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ing arrow between the three secondary subcategories in the conceptual frame- 
work depicting the use of English in the classroom (Figure 6.2). The main 
study aids mentioned, which were used in the classroom during the teaching 
of the subjects, were slides, followed in order of frequency by texts and pages, 
thus demonstrating material that would require reading. This finding wholly 
corresponded with the data provided by the students in the questionnaires, 
who had identified study material provided by their lecturers as the most fre- 
quent source of reading they were required to do in English for their studies. 
The lecturers also made references to “videos”, which implied they would 
provide additional opportunities for listening practice. In fact Ottavio, who 
mentioned using videos, used recordings of himself for further input for his 
course, although it was not clear whether he used these in class or if they were 
made available to the students for consolidation purposes. Dario, however, 
stated he used commercially available videos from news websites during his 
classes. 

The academic skills mentioned by the lecturers comprised “presenta- 
tions”, which were included in this category when lecturers referred to the 
students having to make presentations in class rather than presentations being 
used as a listening activity when they were given by the lecturer or others, 
followed by “notes”, “exercises” and “understand”. As discussed in the pre- 
vious section, some of these findings tended to differ from the findings gen- 
erated from the students’ data analysed from the questionnaire responses, for 
example regarding the prevalence of the use of presentations as a means for 
student participation in the classes. However, the lecturers concurred with the 
students that taking notes was a frequent activity undertaken in their class- 
rooms, which had been the joint third most frequent writing activity chosen 
by the students. The lecturers also provided information to confirm that these 
notes were being taken in English and not in the students’ first languages. 
Fabio affirmed that he had noticed “a couple of them taking notes in English”. 
Claire also made comments about observing her students’ taking notes in Eng- 
lish and how she felt it would be problematic for them to take them in a lan- 
guage other than English. She stated “the notes I have seen are in English... I 
mean I know it’s just too difficult to take notes in another language”. Rodion 


also commented in a similar manner: 
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... take notes and 99% of them take [them] in English. I have seen some attempts to 
do it in national languages but without any success [and] usually it expires after 


one day. The second day it’s already in English. It’s really difficult to translate. 


Consequently, both the students and the lecturers agreed that taking notes in 
English in the classroom was a frequent activity, which has been noted by 
others (Boyle and Forchell, 2014; Siegel, 2018). 

As far as the other non-language categories in the “Use of English” con- 
ceptual framework are concerned, therefore, the main academic skills that 
were mentioned by the lecturers were being able to give presentations and 
take notes in English. The main study aids that were used in the EMI classes 
were principally materials that needed to be read, and comprised slides and 
texts from various sources. The findings from this conceptual framework gen- 
erally corresponded with the findings from the questionnaires although there 
were the divergences regarding students giving presentations and the use of 


essay writing, as has been discussed. 


6.1.2.2 Findings from the “Problems encountered in the target situation” 
conceptual framework and implications 

The interviews also sought to investigate the problems encountered by the 
students and lecturers in the target situation and as such functioned also as a 
present-situation analysis, like parts of the questionnaire. The “Problems en- 
countered in the target situation” conceptual framework depicted in Figure 
5.10 presented the findings from this analysis and divided the problems into 
the three main categories, language problems, content problems and problems 
related to classroom management. Again this visualisation appears in Figure 


6.3 for ease of reference. 
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The language problems were further divided into the lecturers’ problems and 
the students’ problems as well as the subcategory “no problems”. This third 
category was added as several lecturers had referred to areas where there 
were no problems, especially when referring to their own use of English as a 
medium of instruction. Dario, when referring to his own language problems, 
stated “Usually no I don’t have any problem when I lecture absolutely no”. 
Benno also answered in the negative when asked about any problems he may 
have had lecturing, “I don’t think I have any particular problem”. Rodion, the 
mathematics lecturer, also felt he had no problems with the language. The fact 
that some of these comments came from the lecturers of mathematics and sci- 
entific subjects mirrors findings from a recent study conducted with partici- 
pants from Austria, Italy and Poland that examined higher education teach- 
ers’ attitudes to English-medium instruction. This study showed that some 
lecturers felt they had few or even no problems teaching in English since 
“there was a belief that teaching science and maths was easy and required 
little language” (Dearden & Macaro, 2016, p. 471). However, the study did 
also highlight the many language problems lecturers experienced, and which 
will be referred to in due course. 

The comments that were coded into the “no problems” subcategory, how- 
ever, almost always related to the lecturers not having problems with the lan- 
guage. It was only Riccardo who actually referred to the students not having 
any problems with the language. He explicitly mentioned three times in his 
interview that in his view the students did not have any problems, even going 
as far as stating at one point, “they speak very well, they haven’t problem 
about the English no no absolutely no”. However, he was also the only inter- 
viewee who stated that he felt that his English was of a lower proficiency level 
than that of the students: “my level is not so high ... in my opinion the back- 
ground of this student is higher than mine”. Consequently, because he was 
the only lecturer to admit that his English skills were perhaps lower than those 
of some of the students he was teaching, this could imply that he was unsure 
what English language level would actually be required by his students. The 
fact that lecturers who teach in English may be unaware of the language levels 
needed by their students to follow an EMI course has been noted elsewhere 
(Dearden & Macaro, 2016, p. 472). However, another reason for Riccardo’s 
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statement could be due to the fact that “Italy lags behind other European coun- 
tries in terms of multilingualism and in particular the learning of English” 
(Costa & Coleman, 2013, p. 6). The Eurobarometer survey conducted by the 
European Commission (2012) showed that Italy was ranked second from last 
among 27 EU nations regarding self-reported proficiency in a second lan- 
guage. In fact only 38% of Italians surveyed claimed to be able to speak at least 
one foreign language, compared to the EU average of 54%. There is therefore 
a tendency for Italians to admit to a low level of competence in other lan- 
guages, especially English, which was documented by Dearden & Macaro in 
their study (2016), even if perhaps this is only a perception. Thus, this general 
lack of confidence in their foreign language skills shown by Italians could be 
a reason for Riccardo’s statement that his English was lower than that of some 
of the students. 

The lecturers’ problems in the “Language problems” subcategory, as can 
be seen in the conceptual framework in Figure 6.3, were divided into two fur- 
ther categories. The first and smaller of the two was coded “content related” 
problems, and the other, larger category was related to the fact that the EMI 
lecturers were all non-native speakers of English. The content-related category 
contains the in vivo coded expression “dumb down” which was mentioned by 
Ottavio, who stated “I tend to dumb down, downgrade my English” in order 
to make his English more comprehensible for the students. This implies, there- 
fore, that he was taking measures to simplify his language, which is a strategy 
that can be used to modify the comprehensible input (Krashen, 1982) for L2 
learners and can “involve reduction or regularization of surface forms (restric- 
tive simplification) or, alternatively, an increase in surface forms for the sake 
of elaboration or clarification (elaborative simplification, as in the case of rep- 
etitions and paraphrases)” (Chaudron, 1983, p. 439). Although the exact type 
of simplification to which Ottavio was referring cannot be inferred from his 
statement above, the fact that he felt he needed to simplify his language im- 
plies the language proficiency of the students was not sufficient to engage 
fully with the content of his subject. The concern that teaching in learners’ L2 
(or even in the students’ L3 in some cases) can have an adverse effect on the 
teaching and learning of content in EMI contexts has been reported by others 
(Coleman, 2006; Wilkinson, 2012; Guarda & Helm, 2016; Briggs et al., 2018) 
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because of low language proficiency either of the lecturer or the students. In 
this particular case, Ottavio was aware that the language he was using might 
not be comprehensible for all the students due to their language proficiency, 
but using terminology that might be less problematic to understand could 
have had a negative impact on these students’ content learning. In Briggs et 
al.’s recent study of EMI teachers’ beliefs, it was observed that “students with 
a deficit of English language proficiency were likely to be negatively affected 
with regard to their subject discipline content learning” (2018, p. 687). How- 
ever, it has been noted elsewhere that “teachers’ limited language skills nega- 
tively affect elaboration, explanation and classroom interaction ... which do 
not encourage ... student participation” (Drljača Margi¢ & Vodopija-Krsta- 
novic¢, 2018, p. 32). Although there is little evidence that language proficiency 
problems experienced by students in this study had a direct impact on their 
content learning, various suggestions that a relationship did exist were made 
by the lecturers in the interviews. Consequently, this tentative relationship is 
expressed in the conceptual framework in Figure 6.3 with a double-headed 
dotted arrow between the main subcategories of “Language problems” and 
“Content problems”. 

The other language problems that lecturers mentioned they experienced 
were all related to their not being native speakers and Enzo even commented 
“Well clearly it’s not my mother tongue so that’s the basic problem”. One of 
these problems related to vocabulary, where lecturers mentioned that their 
“vocabulary is limited” (Oscar) and that they have difficulties producing the 
correct terminology (Riccardo, Oscar, Claire), whether that be general words 
or specific words. However, both Dario and Enzo felt that the production of 
specific terminology was actually easier in English than in Italian, given the 
status of English as the international academic community’s lingua franca 
(Crystal, 2003; Mauranen et al., 2010). Dario stated clearly “Iam more used to 
teaching in English, talking in English about finance than in Italian”. This find- 
ing from the interview data, which suggested that vocabulary was more of a 
weakness than a strength, resonates with some of findings from a subsequent 


study at the same university (Quick, 2021). It also corresponds with findings 
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from a broader survey of Italian lecturers in EMI contexts where some lectur- 
ers felt confident about their use of vocabulary in their teaching, but many 
others felt that it was a problem (Helm & Guarda, 2015). 

Other problems relating to the lecturers’ not being L1 English speakers 
were their lack of fluency (Fabio) and their pronunciation (Dario, Enzo, Ivan, 
Riccardo). Fabio stated that he encountered a lack of fluency when he first 
started teaching in English but that “it is a matter of warming up” and through 
continued use of English the problem had lessened. Pronunciation problems, 
however, were mentioned by several lecturers, often referring to their per- 
ceived marked Italian accents, such as Enzo who stated “my English is with 
an Italian accent”. Concerns about pronunciation were also frequently ex- 
pressed by the Italian lecturers surveyed in the study undertaken by Helm & 
Guarda (2015). Particularly, in their study, some lecturers expressed concern 
because their pronunciation did not follow native-speaker norms, which was 
also apparent from the data collected in my study. Dario, for example, stated, 
“I try to kind of mimic the British accent and not talk too much like an Italian”, 
and Ivan affirmed “I have a non-British accent”. Claire, despite not referring 
to any issues about her pronunciation, did mention her teaching assistant, 
who “has a nice British accent”. The prevalence of references to a “British” 
accent, although perhaps unusual due to its specificity, certainly originated 
from my prior relationships with all the interviewees and that the interviews 
conducted in this study can most definitely be classed as “acquaintance inter- 
views” (Garton & Copland, 2010). Although data generated in acquaintance 
interviews are not “in any way more valid (or invalid) than data collected in 
more traditional social sciences interviews” (Garton & Copland, 2010, p. 548), 
it is likely that the references to a British accent would not have been so fre- 
quent had I been a different researcher. 

The “Language problems” category in the “Problems encountered in the 
target situation” conceptual framework depicted in Figure 6.3 also comprised 
the students’ problems, and this was unsurprisingly the category that covered 
the most areas in the conceptual framework, given the focus of the target sit- 
uation analysis and therefore the questions asked in the interviews. The stu- 
dents’ language problems were subdivided into “vocabulary” and “skills” 
and were coded as purely language problems in the conceptual framework 
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using red as the outline for the language problems and using one arrow orig- 
inating solely from the “Students’ problems” subcategory. The lecturers also 
referred to other problems experienced by the students, which were coded as 
“cognitive” and “affective” problems, but because these could not be classed 
as pure language problems but clearly influenced the language aspects, they 
were connected by more than one arrow. Therefore, some lecturers reported 
that students manifested problems that were coded as “cognitive”, but these 
tended to originate from issues regarding the content of the classes rather than 
the language used. 

Vocabulary problems were mentioned by several lecturers, referring 
mainly to students having difficulty with the specific lexis used for the subject. 
Oscar, the lecturer of Philosophy, mentioned the problems the students expe- 
rienced with the specific terminology used in philosophy on several occasions 
and at one point discusses the problems students faced at length: 


I don’t know how much they read the text but even in class we read the text together 
and they don’t ask the words they don’t know. I have to ask them, do you know 
this word and they're uh? and then I explain it but it’s strange, I don’t know why 


they don’t do that because maybe they are afraid ... 


As mentioned previously, philosophy is one of the more language-heavy sub- 
jects that is taught in English at the Faculty, and so the fact that students ex- 
perienced problems with specific terminology would be understandable. 
However, he went on to admit that perhaps it was the subject of philosophy 
itself that was the challenge, rather than the fact that it was taught in English 
when he stated “there is a very specific language I have to rely heavily on 
single words, analyse them and so on and this is something generally they are 
not confident with in any language”. Other lecturers mentioned problems 
with specific terminology, including Ottavio, who stated “Sometimes they 
have problems knowing the right words, depends on the topic but some topics 
they have problems knowing the right English words”. Ivan also recognised 
that students had problems in his Financial Analysis course and referred to 
how he tried to mitigate these problems: “I try to simplify the scientific lan- 


guage so that I am sure they have come across the words and I am using in 
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the exam questions during the course”. In contrast, Fabio stated that the stu- 
dents did not have a problem understanding technical terms but in fact had 
problems understanding more general words. He gave the following example: 


I’m there to explain the technical terms but sometimes either in the exam and in the 
class they lack in understanding also basic terms, that is for instance... once I made 
the example of demand and supply in the automobile sector, what happens to the 


sector of tyres. I had questions... what does tyre mean? 


Consequently, although many of the lecturers did not explicitly state that they 
felt students experienced problems with vocabulary, when problems were 
mentioned, they tended to focus on the specific vocabulary needed for the 
subject. This finding corresponds with another study conducted by Evans & 
Morrison (2011a) that investigated the use of English in an English-medium 
university in Hong Kong using a similar approach to that used in my study 
where a questionnaire to students to elicit their perceived strengths and weak- 
nesses in the four skills was administered, although they then conducted fol- 
low-up interviews with a sample of those students. An important general 
theme that emerged from their interview data was “technical vocabulary” and 
in particular, students’ “inability to understand key technical vocabulary” re- 
sulted in problems in reading and listening (Evans & Morrison, 2011a, p. 393). 
However, in their study, it emerged from the students’ accounts that the lec- 
turers of the EMI classes were often “oblivious” to the problems caused by a 
lack of technical vocabulary knowledge (Evans & Morrison, 2011a, p. 393), 
which certainly contrasts with the findings from my study. Moreover, in my 
study, one lecturer referred to the problems students had with general vocab- 
ulary. 

Although some of the lecturers mentioned problems the students experi- 
enced with vocabulary, the main language problems tended to focus on the 
skills. Reference to problems in all the four skills was made, but the skills that 
were regarded as presenting the most problems were speaking and writing. 
Speaking problems were identified by five of the lecturers (Dario, Ivan, Claire, 
Ottavio and Fabio) and comprised difficulties connected to a lack of fluency 
(Fabio). However, most lecturers did not necessarily focus on the speaking 
problems themselves but rather the cause of the problems, which became its 
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own category, “affective problems”, and included “anxiety”, “shyness” and 
“lack of confidence” in the conceptual framework due to the prevalence of 
these aspects being mentioned. Some of the comments from the lecturers con- 
cerning this aspect concentrated solely on students’ lack of confidence with 
spoken English: “I see that they have... they’re not confident with their speak- 
ing” (Claire) and “In class of course some of them are not confident with their 
spoken English” (Oscar) whereas Dario felt that it was the use of English that 
exacerbated an already existing lack of confidence: “well probably some of 
them are shy beforehand but I believe that — the speaking in a foreign language 
is part of the story”. Moreover, Fabio stated “this year for instance I had a very 
low responses to... I don’t know what happened, they were quite shy”, im- 
plying that he felt that it depended on the cohort and that a lack of confidence 
was perhaps not necessarily a regular occurrence. This general problem of stu- 
dents lacking confidence in their speaking has been noted in another study on 
EMI undertaken at the neighbouring University of Trento in northern Italy, 
where the lectures who were interviewed stated their main concern was that 
“their students were reluctant to use English (due to shyness, fear of making 
mistakes, insecurity, etc.)” (Polli, 2021, p. 88). Costa & Mariotti (2020) also 
noted this lack of confidence in their study where they compared Italian stu- 
dents’ attitudes to studying on an EMI programme with those of international 
students. However, in their study, the local students’ lack of confidence was 
significantly more marked than that of the international students. 

Students’ reticence to speak in the EMI classroom is a phenomenon that 
has been observed for some time (Tsui, 1996; Chang, 2010; Soruç & Griffiths, 
2018), and this has been attributed to multiple factors originating from the 
learners themselves, from the methodology used by their teachers and from 
the settings in which the learning takes place. Benno, when mentioning this 
aspect, focused on the fact that although some students seemed to understand 
the subject matter, he felt that it was anxiety about their English proficiency 


that was preventing them from interacting in the classroom: 


sometimes the students who are less confident, not so much in the discipline that is 


being taught but in the language, in English, actually are more shy. 


178 


Findings From the Data Analysis in AR Cycle 1 and Discussion 


The phenomenon of shyness hindering speaking output was also observed in 
the study by Dearden & Macaro where it was the “students’ poor level of Eng- 
lish [that] inhibited their learning, made them embarrassed” (2016, p. 473). 
Indeed, this fear that is often displayed by less proficient language learners 
has shown to increase what is known as “communication apprehension”, 
which refers to an individual’s “fear or anxiety associated with either real or 
anticipated communication with another person or persons” (McCroskey, 
2001, p. 40). This apprehension can lead to learners actively avoiding situa- 
tions where speaking is required, which “deprives learners of the practice that 
they need in order to improve their speaking skills and become confident lan- 
guage users” (Zhang & Head, 2010, p. 2). This aspect was also mentioned by 
Dario, who referred to the fact that he felt students were impeded from speak- 
ing in class due to this apprehension, which then had effect on communicating 
their subject knowledge: 


however for some of them they are handicapped because they know things but they 


are just afraid of speaking out loud. 


Oscar had also identified a similar trait when he spoke about the students’ 
unwillingness to ask him about unknown words, and he also attributed it to 
the students’ anxiety, using the term “afraid”, like Dario above, and so this 
was coded together with the other references to affective problems. Claire was 
another lecturer that commented on this unwillingness to speak out in class, 
although she did not directly attribute it to any of the factors the other lectur- 
ers had mentioned, when she stated: “Sometimes they will act as if they are 
struggling with the content but in fact they just don’t want to answer in Eng- 
lish I think”. Willingness to communicate (WTC) in an L2 differs from WTC 
in a person’s native language since the individual’s communicative compe- 
tence in the L2 is a “powerful modifying variable” (Dérnyei, 2003b, p. 12). 
Studies have found that learners with a lower language proficiency can expe- 
rience greater anxiety when having to speak out in class and thus they can 
demonstrate less willingness to communicate (Thompson & Lee, 2013; 
Thompson & Khawaja, 2016). However, communicative competence and 
WTC are not the same and it has been noted that learners who are competent 
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L2 speakers might avoid communicative situations in the L2 and less profi- 
cient speakers might actively seek opportunities to engage in L2 interaction 
(Dérnyei, 2003b). This aspect is clearly visible in the following extract from 
Claire who stated: 


Some of them answer, some of them are very good and I know they know the an- 
swer, they just stand there and wait for me to ask and then, Paolo what do you think 


and then Paolo gets up with a perfect answer in perfect English 


The fact that learners demonstrate anxiety in an EMI context, which then pre- 
vents them from speaking, has also been attributed to the settings that char- 
acterise EMI. The recently published study by Sorug & Griffiths, which inves- 
tigated an EMI context in Turkey by administering an open-ended question- 
naire to students, found that many difficulties experienced by the students in 
their speaking were due to affective aspects such as shyness and feeling em- 
barrassed. These affective aspects were often attributed to the way the classes 
were delivered, which “were conducted along fairly traditional lecture-style 
lines” (Sorug & Griffiths, 2018, p. 40). Aslan & Thompson (2021) also suggest 
that anxiety about classroom performance could be due to the typical teacher- 
fronted nature of the context they studied. 

Learner reticence has also been attributed to certain methodological prac- 
tices that are often used by the teacher or lecturer in class, particularly in tra- 
ditional teacher-fronted classrooms. These practices include teachers’ intoler- 
ance of silence and thus a shorter wait time, so that the turn is either reallo- 
cated or teachers provide the answer themselves, uneven allocation of turns 
where the more confident students are more frequently called upon, and in- 
comprehensible input where a lack of responses is attributed to learners not 
understanding instructions or questions (Tsui, 1996, pp. 151-154). An example 
of the uneven allocation of turns was provided by Claire when she stated: 


I choose the students I ask. I know about their level and when I see one is struggling, 
I will not, I will ask for help from somebody else and I try not to ask students that I 


know will have difficulties 
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The strategy exemplified in this extract is efficient in advancing through the 
class, especially when there are time constraints to which Claire also makes a 
reference when she states “I have so [many] things to do in the class time that 
I have that I don’t have time”. However, it has been shown that uneven allo- 
cation of turns can make the weaker or shyer students feel neglected and “the 
more they feel neglected, the less willing they are to contribute” (Tsui, 1996, 
p. 154). Consequently, the fact that anxiety and shyness are factors that hinder 
learners’ willingness to speak, even if they do not necessarily always derive 
from a learners’ communicative competence, is clearly demonstrated from the 
data collected from the lecturers in this study. 

Apart from speaking, the other skill that was mentioned most frequently 
by the lecturers as being problematic for the students, and was coded and 
added to the conceptual framework, was writing. These problems arose 
mainly in the exams, which, as mentioned, are all conducted as written exams 
in the Faculty. Oscar, the lecturer for Philosophy stated: 


they write in German or Italian with English words of course. So they don’t know 
what an English sentence is, they don’t know how to connect two sentences, many 
of them... the fact that how a sentence is meant to be connected in order for a Eng- 


lish eye to make sense of what is written 


This comment focuses on problems that originated from syntax and discourse 
features rather than lexical problems since he referred to the students’ diffi- 
culties to create cohesive texts that follow typical English syntactical struc- 
tures. Claire, however, focused on the assignments the students had to pro- 
duce for her course, rather than the writing done in the exam, and how she 


felt students had difficulties expressing their own ideas in English. She stated: 


They are not confident even in their writing because they quote a lot, they copy and 
paste, in first year much more than in third year because third year they know I can 


detect it but in first year the report I see a lot of copy and paste but... 


In this case, their difficulties resulted in a tendency to resort to copying from 
other sources, an academic problem that she ascribed to originating from a 


lack of language proficiency. Another problem for students when writing was 


181 


Integrating Classroom-Based Negotiation Into a Syllabus 


highlighted by Benno, who referred to students’ tendency to write too much 
when answering questions, thus losing coherence in their texts and failing to 


complete the set task adequately: 


they always try to look competent, knowledgeable by writing long answers and I 


say, well the first thing is stay on topic because of course that’s more important 


The comments shown in these extracts, therefore, tended to relate to problems 
regarding specific academic writing skills especially related to connected dis- 
course and syntax. They also concerned difficulties in producing texts that had 
not been copied from elsewhere. Although the inability to produce texts that 
are not copied could stem from a lack of academic skills such as poor refer- 
encing or inadequate citations, or indeed, from a deliberate desire to cheat, it 
could also be due to difficulties with the language, as Claire implied. Indeed, 
“a growing body of research into L2 students’ source-based writing has re- 
vealed language-related problems which may lead to inadvertent plagiarism” 
(Pecorari & Petrić, 2014, p. 275), so it is likely that the students in this study 
were either lacking writing subskills such as paraphrasing and summarising 
in certain cases or perhaps were even lacking specific reading skills. The prob- 
lem for students to produce texts without resorting to plagiarism because of 
their lack of language proficiency has also been addressed by Hyland, who 
found that, “After they mentally compare their texts with target ‘expert texts’, 
they may feel so overwhelmed by the distance between what they are ex- 
pected to achieve and what they feel capable of doing, that plagiarism seems 
the most realistic strategy” (2001, p. 380). Consequently, Claire’s assertion that 
her students’ attempts to copy from other sources in their assignments origi- 
nated from language difficulties rather than from any desire to act dishonestly 
would seem reasonable. 

However, in the interviews, many lecturers did not refer to writing prob- 
lems, not necessarily because they were no problems, but because their 
courses did not require a significant amount of writing, if any, due to the sub- 
ject matter, which was mainly financial or mathematical. Indeed, Fabio stated 
clearly that writing was not needed as he used multiple choice questions in 
his exam. Writing was therefore considered to be a language problem for the 
students especially in the more language-heavy courses, such as Politics and 
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Accounting, and tended to comprise aspects related to specific academic writ- 
ing skills, as discussed. 

Other problems that were mentioned by the lectures, although the in- 
stances of these references were far fewer than for speaking and writing, con- 
cerned reading. Ottavio, the lecturer for the Information systems and data 
management course, referred to students’ problems in reading, particularly 
when confronted with unknown lexical items that had an impact on under- 


standing written texts: 


I have the impression that some of them don’t read carefully. Others read but don’t 
understand — they skip some key words in my sentence which completely changes 
the meaning maybe or which underlines what they must do. So understanding the 


text is the major English skill concerning my exam. 


His comment here concerns the fact that some students had problems with 
academic reading skills, particularly inference of unknown lexical items, 
which subsequently led to comprehension problems that had a negative im- 
pact on the tasks they had to complete for his exam. Problems concerning lis- 
tening were also seldom mentioned by the lecturers, but Dario stated that he 
experienced problems when he brought authentic video material into his 


class: 


I wanted to watch together with the students some videos from the FT or from the 


Economist and they were completely lost. 


Although he did not explicitly say what he felt caused these problems, he did 
continue by saying, 


I understood that the only way to get them to understand something was to first 
show all the footage and then talk about what was told and then show it again — 


piece by piece discussing what we had just heard 


The fact that he had to show the videos in segments and then discuss each 
section with the class implies that students most definitely experienced com- 
prehension problems, which in lectures can often be due to either decoding 


183 


Integrating Classroom-Based Negotiation Into a Syllabus 


problems, where students typically have difficulties recognising what has 
been said, or problems with spoken discourse structures. Clearly, these diffi- 
culties can be compounded by the accent and speed of speaking of the 
speaker(s), all of which indicate language problems. However, it has been 
noted that difficulties in understanding spoken language used in lectures can 
also be due to learners’ “insufficient knowledge of the specialist subject” (Jor- 
dan 1997, p. 179). Therefore, although this particular aspect was coded as a 
problem with the skill of listening for this analysis, it could have equally de- 
rived from students’ lack of subject knowledge and would have therefore been 
coded as “content problems”. 

As has been discussed in this section so far, therefore, although the lec- 
turers recognised that problems were experienced by their students in all four 
skills, these problems mainly concerned the productive skills; writing and 
speaking. However, the problems students encountered in the target situa- 
tion, as has been mentioned previously, not only related to language aspects. 
Problems also arose from the content of the courses and the conceptual frame- 
work depicted in Figure 6.3 shows this category and its main subcategory, 
“cognitive problems”. The cognitive problems mentioned were related to a 
lack of understanding of the subject matter and aspects linked to attention 
problems. References to problems concerned with understanding occurred 
multiple times in the interviews. In the following statement from Dario, he 
referred to a strategy that he felt would be useful to alleviate students’ diffi- 
culties in understanding the subject matter of his course: “I think some multi- 
media device like some video or — some recording of speeches — could improve 
their understanding of some topics”. However, as just discussed, he also rec- 
ognised that when audio and video support were used in his classes, some 
students struggled with the comprehension of those videos. 

Ottavio, on the other hand, commented on a cognitive problem that also 
affected the students’ ability to undertake the tasks he used in class when he 
stated: 


Others are attention problems, so they don’t pay enough attention to what I write 


or they don’t have any idea that that thing may be important. So I would say lan- 


guage problems only a minority 30% of the cases. 
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Here, not only did he focus on the fact that the students did not seem to be 
able to distinguish main ideas from supporting ideas, a common problem re- 
ported by students and their teachers in English-language classrooms (Jordan 
1997), but also that some seemed to become distracted and so did not pay at- 
tention. Becoming distracted in class was also a difficulty that was reported 
by the students surveyed in the study conducted by Soruç & Griffiths (2018, 
p. 46), which they suggest might be due to multiple factors including students’ 
inadequate language levels for courses taught in English. 

The “Problems encountered in the target situation” conceptual frame- 
work also comprised the other main category of problems mentioned by the 
lecturers, which concerned aspects connected to classroom management. This 
subcategory included effects of the large class sizes and traditional teaching 
approach, especially affective factors such as anxiety, as discussed previously. 
There was also reference to time problems, particularly by Oscar and Claire, 
who both referred to not having enough time for their classes or course. 

From the analysis of the “Problems encountered in the target situation” 
conceptual framework, therefore, the most useful and relevant findings that 
helped to answer the first research question were that the students, when they 
experienced language problems, were most likely to have problems with their 
speaking, to a lesser extent their writing and also with vocabulary. The speak- 
ing problems tended to be influenced by affective factors, such as anxiety and 
shyness, which prevented many students from engaging in meaningful spo- 
ken interaction. These affective factors in turn tended to be influenced by mul- 
tiple factors, ranging from students’ lack of language proficiency to the class- 
room environment but also most probably by some of the teaching methodol- 
ogy employed by the EMI lecturers. Students’ writing problems were gener- 
ally encountered by the lecturers of the more language-heavy subjects such as 
Politics and Accounting, and concerned specific academic writing skills such 
as producing connected discourse, demonstrating cohesion as well as general 
problems with syntax. Vocabulary problems were also reported by many lec- 
turers and although they tended to be associated with a lack of knowledge of 
specific terminology, problems with basic or general vocabulary were also re- 
ported. Other problems that were not specifically language problems, but 
which affected the students’ learning included content problems, especially 
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lack of specific subject knowledge, and factors associated with the classroom 


environment. 


6.1.2.3 Findings from the “Lecturers’ approach to language difficulties” 
conceptual framework and implications 


The third conceptual framework that emerged from the data analysis of the 
interviews was entitled “Lecturers’ approach to language difficulties”. This 
was described in Section 5.5.3.3 and is reproduced below in Figure 6.4 for ease 


Lecturers’ approach 
to language difficulties 
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Fig. 6.4 — The “Lecturers’ approach to language difficulties” conceptual framework 


The data from this superordinate category do not necessarily provide many 
useful additions in order to answer the first research question of this study, 
but an interesting finding that did emerge from this conceptual framework 
was the approach lecturers took when encountering language errors in stu- 
dents’ written work. All the lecturers who made a reference to the language 
mistakes made by their students stated clearly that they did not take these 
mistakes into consideration when marking their exams. Dario stated quite 
clearly “Of course I don’t take into account language mistakes in the exam”, 
and this sentiment was echoed by Rodion, who declared “The issue is that 
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even if their English is sometimes terrible it doesn’t affect the final mark; I still 
don’t take marks off”. Oscar also took a similar position and said: 


I tell them I don’t penalize them, I don’t take away points from them and that I do 
my best to understand what the person was trying to say and if it’s fine then they 


get exactly the same mark as somebody who wrote it in Shakespeare English 


This fact of not considering the students’ English proficiency when marking 
exams has been noted elsewhere. Evans & Morrison found in their study of an 
EMI context in Hong Kong that “professors evidently took little or no account 
of the quality of students’ English when assessing written and spoken work” 
(2011b, p. 207), which in the students’ views, questioned the very existence of 
the EAP course that was also investigated in their study. 

However, in my study the fact that the EMI lecturers seemed to ignore 
the students’ English proficiency and even employed various strategies in or- 
der to accommodate the deficiencies in the students’ English-language output 
demonstrates their apparent unwillingness to take any responsibility for the 
students’ language development. Their interest seems to focus purely on the 
students’ output from a content perspective. This attitude has been reported 
in Dearden & Macaro’s study, where the majority of the EMI lecturers who 
responded to their survey felt it was not their responsibility to develop the 
students’ language competence (2016, p. 478). However, Dearden & Macaro 
also quite correctly question whether content lecturers in EMI contexts should 
take responsibility for students’ language development, and this question so 
far does not seem to have been answered conclusively. What is clear, though, 
is that the findings from my study demonstrate that the lecturers who were 
interviewed and expressed an opinion on this subject did not believe provid- 
ing language teaching was their duty. Moreover, Claire was the only lecturer 
who admitted that she penalised the students for language errors stating “if 
what they write is the contrary of what they wanted to say, I will take off a 
mark or a grade”, but she also abdicated responsibility for the students’ mis- 
takes in English, stating “I’m not correcting the English”. Dearden & Macaro 
felt this unwillingness to take any responsibility for the students’ language 
development in EMI contexts that they also documented could have contrib- 
uted to the low student motivation levels about which those same lecturers 
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had been complaining (2016, p. 473). Given that in my study many of the lec- 
turers seemed to suggest it was often affective factors such as lack of confi- 
dence and shyness that contributed to students’ language problems, as was 
also the case in the Dearden & Macaro study, it would seem reasonable to 
conclude that some of the language problems experienced by the students in 
my study might have actually originated from the lecturers’ reluctance or in- 
ability to engage with the pedagogical aspects of teaching in an L2. Indeed 
Dearden & Macaro recommend that 


EMI teachers need to know how to modify their input, assure comprehension via 
student-initiated interactional modifications and create an atmosphere where stu- 
dents operating in an L2 or L3 are not afraid to speak. (Dearden & Macaro, 2016, 
p. 479) 


Clearly, the focus of my study is the ESAP course I was and still am teaching 
and not the EMI courses offered by the other academic staff at the Faculty. 
However, in light of the affective problems that seemed to have been caused, 
at least in part, by some of the pedagogical practices used in the EMI classes, 
introducing a pedagogical approach in the ESAP classes that would enable 
students to operate in an environment where anxiety levels were reduced be- 
cause there were more opportunities for student-initiated interaction thus be- 


came a clear aim for this study. 


6.1.3. Research Question 2: 
Discussion of findings from quantitative analysis 


The questionnaire not only attempted to collect data that could answer the 
first research question, but also aimed to investigate whether there was a rela- 
tionship between how much practice students stated they engaged in and 
their language proficiency levels, in order to test the hypothesis as to whether 
there was a relationship between the amount of practice of a skill undertaken 
and the level of proficiency reported. As discussed in Section 5.3.5 of the 
previous chapter, Pearson chi-square tests were conducted to test whether 
there was a relationship between some of the variables that had been coded 
from the quantitative data collected from the questionnaires to the students 
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and as discussed in that same section, there were significant relationships be- 
tween the frequency of practice reported by the students and their self-re- 
ported level of difficulty. The data that were reported in that same section 
referred to all the responses from the three questionnaires with a total of 365 
responses and concerned the relationship between frequency of practice and 
self-reported level of difficulty for the skill of writing. 

The data for the first questionnaire that was conducted in AR Cycle 1, 
however, which counted only 151 responses, had to be manipulated in order 
to generate meaningful responses given that some of the cells had fewer than 
five results given the smaller total number of responses. Chi-square cross tabs 
analyses were then run for each skill to investigate whether there was any 
relationship between the frequency of practice and self-reported level of 
difficult for those skills. The p values of these analyses can be seen in Table 
6.10. 


Table 6.10 — Relationship between frequency of practice and self-reported difficulties — all four 
skills from all three questionnaires 


Questionnaire 1 Questionnaire 2 Questionnaire 3 Aggregated results 


p value p value p value p value 
Reading 603 113 .000 .084 
Listening 485 .277 .142 099 
Writing .048 .001 .001 002 
Speaking 442 .026 .044 .002 
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As can be seen from the table above, the chi-square test analyses showed over- 
all a significant relationship between the students’ amount of practice and 
their self-reported level of difficulty in that skill only for the productive skills. 
Apart from the results for speaking in questionnaire 1, all the other results for 
speaking and writing were p < .05, which can indicate there is a significant 
relationship between the two variables. From these p values, it can also be seen 
that the aggregated results of the three questionnaires tended to show a more 
significant relationship than the results from the individual questionnaires, 
which clearly had far fewer responses. Given the disparity in the results gen- 
erated by the individual questionnaires in this analysis, it is difficult to state 
that generally there was a significant relationship between the amount of prac- 
tice the students stated they undertook and their self-reported levels of diffi- 
culty in the skills. However, the aim of these analyses was not to generalise 
beyond the context under study but to investigate the responses provided by 
the sample of students who had answered the questionnaire and to draw 
tentative conclusions from the findings for those students as well as 
subsequent students. Consequently, from these analyses it can be seen that 
there does seem to be a relationship between how much the students say they 
practise a skill and their level of difficulty in that skill, but only for the 
productive skills. 

This relationship can be further observed through analysing the trends in 
the data. In Table 6.11 below, the data concerning the students’ frequency of 
practice and their self-reported level of difficulty in the skills are reproduced 
in percentages and refer to the first questionnaire that was administered in AR 
Cycle 1. 
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Table 6.11 — Percentages showing relationship between frequency of practice and self-reported 
difficulties — all four skills questionnaire 1 (n=151) 


Reading very difficult quite difficult quite easy very easy 
re 0.0% 0.0% 52.4% 47.6% 
2-3 times a week 0.0% 4.5% 61.4% 34.1% 
Every day 0.0% 4.7% 44.2% 51.2% 
Listening very difficult quite difficult quite easy very easy 
renee er 0.0% 13.0% 69.6% 17.4% 
2-3 times a week 0.0% 17.2% 58.6% 24.1% 
Every day 0.0% 10.0% 57.1% 32.9% 
Writing very difficult quite difficult quite easy very easy 
Once aweekor 5.3% 44.0% 42.7% 8.0% 
2-3 times a week 8.3% 27.1% 50.0% 14.6% 
Every day 14.3% 50.0% 32.1% 3.6% 
Speaking very difficult quite difficult quite easy very easy 
ere 11.1% 27.8% 52.8% 8.3% 
2-3 times a week 2.1% 33.3% 45.8% 18.8% 
Every day 6.5% 19.4% 25.8% 48.4% 


There is an evident general trend that demonstrates that the more students 
say they practise a skill, the easier they feel that skill is for them and this holds 
for the two receptive skills and speaking, but not writing. While those who 
practise writing only once a week or less are almost equally split between con- 
sidering writing difficult or easy, with 49.3% and 50.7% respectively, the ma- 
jority of those who practise writing every day still find it difficult (64.3%). 
However, the trend for speaking shows a different pattern: while just over 
60% who practise speaking only once a week or less find it easy, almost three 
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quarters of those who state they speak English every day find it easy. Conse- 
quently, although the data presented in Table 6.11 could not be analysed using 
the cross tabs due to many cells registering a 0 value (for example all the “very 
difficult” responses for reading and listening had no value as can be seen in 
Table 6.11), they still provide useful information concerning the frequency of 
practice and students’ self-reported skills levels. 

The questionnaire also sought answers to the second research question by 
further investigating the self-reported levels of the students. Questions 21-24 
were designed to elicit more specific information about what levels the stu- 
dents felt they possessed at the time they took the questionnaire in the four 
skills and what level was needed to study at the Faculty in each skill. The 
responses were presented as adapted descriptors from the CEFR in order to 
extract these data. These questions also aimed to address some of the weak- 
nesses of questionnaires, namely self-deception, where respondents might de- 
ceive themselves as to their strengths and weaknesses, and the fact that often 
respondents will display “a somewhat superficial and relatively brief engage- 
ment with the topic’ (Dérnyei, 2007, p. 105). Indeed, the “I can, I have to” 
questions were deliberately designed to appear in the second half of the ques- 
tionnaire in an attempt to address the problem that questionnaire responses 
can be deemed unreliable precisely due to a lack of engagement with the topic. 
If the respondents had succeeded in progressing all the way to Questions 21- 
24, this would imply that they were engaging with the questionnaire on more 
than just a superficial level. Moreover, the use of more detailed descriptors 
instead of just requiring one of four responses on a scale, which was required 
in the previous question concerning level of difficulty, required a greater en- 
gagement with the questions and their responses. It was hoped that this 
greater engagement would lead the students to provide more accurate infor- 
mation about their language proficiency and help to mitigate any desire on 
their part to underestimate or indeed overestimate their level given that I was 
fully aware that my dual role as their past, current, or future teacher and the 
researcher in this study might influence the data being collected. 

The cross tab analyses of the data can be observed in Table 6.12, which 
were generated after data manipulation, again in order to avoid cells with 


fewer than five values. Consequently, the levels from A1 to B2 were grouped 
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and then the C1 and C2 levels were grouped so that only two proficiency lev- 
els were used in the analysis. The two groups were chosen given the target 
level of the ESAP course, which was C1, and the fact that students were sup- 
posed to have a B2 level to access the course. As can be seen very clearly from 
the percentages in Table 6.12, there is an obvious trend where the more fre- 
quently students say they practise a skill, the higher the level of proficiency 
they report to have in that skill. The findings for all four skills show that over 
60% of those who state they practise the skill every day for all the skills report 
being at the C1 or C2 level. 


Table 6.12 — Percentages showing relationship between frequency of practice and self-reported 
levels — all four skills 


Reading A1-B2 C1+C2 
Once a week or less 51.2% 48.8% 
2-3 times a week 38.5% 61.5% 
Every day 21.1% 78.9% 
Listening 

Once a week or less 51.0% 49.0% 
2-3 times a week 36.7% 63.3% 
Every day 30.6% 69.4% 
Writing 

Once a week or less 59.2% 40.8% 
2-3 times a week 45.2% 54.8% 
Every day 37.9% 62.1% 
Speaking 

Once a week or less 65.1% 34.9% 
2-3 times a week 44.8% 55.2% 
Every day 30.9% 69.1% 


As all the cells contained more than five results, the chi-square tests were able 
to be run and the p values for all four skills showed that the relationship be- 
tween frequency of practice and the self-reported proficiency levels of the stu- 
dents based on the CEFR descriptors was much more significant than for the 
previous analyses that just used the level of difficulty four-item scale. 
Consequently, from these analyses there is an evident shift towards the 
higher CEFR levels the more frequently the students state they practise the 
skill. Moreover, the chi-square tests reveal a significant relationship between 
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the frequency of practice and the proficiency of the students so that the more 
they say they practise the skill the higher their proficiency level. Although 
these findings hold for all the four skills, they are more significant for the pro- 
ductive skills. Therefore, the use of the more detailed descriptors in questions 
21-24 provided more significant relationships between language skill profi- 
ciency and frequency of practice which corroborated to a degree and enhanced 
the findings regarding their self-reported difficulties. 


6.2 Summary Discussion of Merged Findings From Data 
Analysis in AR Cycle 1 


The findings that were generated in the data analysis and the subsequent 
merging of the data from the two datasets in order to achieve triangulation 
were used to help answer the first two research questions of this study. These 
questions were formed in order to provide a framework for the redesign of 
the existing syllabus for the ESAP course that is the focus of this volume. 

The findings as discussed in this chapter showed that the English-lan- 
guage skills needed by economics students at this trilingual university as per- 
ceived by the students themselves were all four of the skills. However, in or- 
der to provide as meaningful an answer as possible to this first research ques- 
tion, the word “needed” requires defining. If “needed” is defined as meaning 
frequency of use, the results from the students demonstrated unequivocally 
that they had to use all four skills on a regular basis and that the use of these 
skills was multifaceted and multi-purposed. Although the students reported 
that they tended to have more frequent contact with the receptive skills, par- 
ticularly reading, they still had regular contact with writing and speaking, 
with one in five reporting they had to write or speak English every day. How- 
ever, the data from the questionnaires did not provide any indication as to 
how useful this practice was nor, in the case of speaking, how prolonged any 
interactions were. As such, therefore, students who stated they practised 
speaking English every day might have only engaged in brief transactional 
exchanges but they might equally have spent more prolonged periods inter- 
acting with their peers in class or their lecturers in class or in office hours. 

The data from the students revealed that there were many different 


purposes for skills use, from reading material provided by the lecturers, 
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taking notes in lectures to interacting with their lecturers in class and in office 
hours. In summary, the data showed that all four skills were required to be 
used in the classroom with differing degrees of importance and regularity, but 
the exams tended to be dominated by writing, although reading was also 
required. 

If “needed” is understood to refer to proficiency level, however, the stu- 
dents generally felt that their English proficiency levels were at least the B2 
level in all the skills, but sometimes quite significantly below the C1 level. The 
students consistently rated their receptive skills at a higher proficiency level 
than their productive skills, although there were divergences between 
whether speaking or writing was the weaker skill depending on the results 
generated by the various analyses that were run. Consequently, from a profi- 
ciency perspective, the productive skills demonstrated more significant defi- 
cits than the productive skills and as such were needed to be improved more 
than the receptive skills. 

The findings that were extrapolated from the interview data with the ten 
EMI lecturers corroborated much of what was generated by the questionnaires 
with the students. The lecturers agreed that the four skills were needed in their 
EMI courses and from a frequency perspective, the most significant appeared 
to be speaking. As far as their exams were concerned, however, it was con- 
firmed that writing was the skill that was most frequently required given that 
the exams are all written exams at the Faculty. The qualitative data from the 
lecturers provided an added perspective to the frequency of use as it became 
clear from the data that speaking was the skill that was most often referred to 
when discussing the use of English in the classroom; a feature that could not 
be inferred from the questionnaire data. However, these references to the 
skills also comprised references to the problems that the lecturers felt the stu- 
dents experienced, and very few mentioned problems related to reading and 
listening, which were the skills the students reported practising the most. 

The data from the lecturers also generally confirmed the data from the 
students regarding for which purposes the skills were used, although there 
were variances with regard to essay writing and students giving presenta- 


tions. While not being able to provide information about the actual language 


195 


Integrating Classroom-Based Negotiation Into a Syllabus 


proficiency levels of the students, the lecturers were able to provide infor- 
mation about the problems they felt the students experienced in their classes 
and exams. Although there were many language problems mentioned, prob- 
lems also concerned other aspects that had an impact on the teaching and 
learning, such as affective and cognitive issues as well as issues connected to 
the specific content of the subjects taught in English. The findings showed that 
the lecturers felt speaking English in class was the most significant problem 
for the students, followed by writing problems encountered predominantly in 
the exams, as well as problems connected to an insufficient command of the 
technical and sometimes also basic vocabulary needed to study the various 
subjects in English. 

The data from the interviews also produced findings concerning the 
origin of the students’ language problems, and significantly one of the main 
reasons given for the students’ perceived lack of speaking proficiency was not 
necessarily their overall communicative competence but their unwillingness 
to speak, especially due to affective aspects related to anxiety. Although the 
lecturers did not offer any reasons for this general reticence to speak, the find- 
ings from the interview data show that all the courses that used English as a 
medium of instruction were taught as traditional lectures, almost undoubt- 
edly due to the relatively large class numbers, but also very possibly due to 
the academic traditions of university teaching in Italy where a traditional lec- 
turing style has been reported as being extremely prevalent (Costa & Cole- 
man, 2013). This traditional teacher-fronted lecturing style used in all the clas- 
ses that were investigated in my study was observed by Costa & Coleman in 
their study to provide students with few if any meaningful opportunities for 
interaction, a phenomenon that has been recognised in other studies of EMI 
classrooms (Zhang & Head, 2010; Wilkinson, 2012; Dearden & Macaro, 2016; 
Soruç & Griffiths, 2018). As a consequence, although most of the lecturers in 
my study stated that the students needed to speak in their EMI classes, there 
were probably generally few opportunities for speaking in those classes and 
they would almost certainly not have allowed many opportunities for 
student-initiated interaction or more prolonged student-student interaction 


given the class sizes and the traditional lecturing styles employed. 


196 


The Negotiated Syllabus: Implementation, Evaluation and Discussion 


In sum, as far as frequency is concerned, both the students and the lecturers 
agreed that all four skills were used regularly in the classroom with a di- 
vergence in the findings concerning which skill was reported to be the most 
frequently used. Both the students and lecturers, however, agreed that writing 
was the main skill that was needed for the exams. As far as proficiency in the 
skills was concerned, the productive skills were rated as being the weaker 
skills by the students themselves, and the findings from the interviews with 
the lecturers tended to correspond with this view. 

These findings not only provided valuable information to inform the first 
research question, but they also provided input for the second research ques- 
tion, which concerned what skills practice should be maintained or enhanced 
in the syllabus. The cross tab analyses of the questionnaire responses concern- 
ing frequency of practice and level of difficulty showed that there was a gen- 
eral trend towards fewer difficulties in the skills that were practised more. The 
cross tab analyses of frequency and proficiency levels as presented in the I can 
statements also showed a trend towards a higher proficiency level the more 
the skills were practised. The chi-square tests that were run on the same vari- 
ables showed a significant relationship between the frequency of practice the 
students stated they undertook and particularly their self-reported levels in 
the skills so that the more frequently they stated they practised a skill, the 
higher their proficiency was in that skill. These findings were particularly sig- 
nificant for the productive skills. 


7. The Negotiated Syllabus: 
Implementation, Evaluation and Discussion 


7.1 Introduction 


This chapter will present how the findings from the data analysis in AR 
Cycle 1 contributed to the redesign of the ESAP course’s syllabus, which was 
then implemented in AR Cycle 2. The newly adapted syllabus that was used 
in AR Cycle 3 will be illustrated and a further problem that arose concerning 
students’ lack of regular attendance and the solution adopted will also be dis- 
cussed. 
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7.2 Research Question 3: 
How Can a Predominantly Product Syllabus That Is 
Skills Based Benefit From the Integration of a 
Process Approach to Syllabus Design? 


The findings that were generated from the merged datasets in AR Cycle 1 pro- 
vided the focus for the redesigned syllabus and as such contributed in part to 
answering the third research question. The existing syllabus, which was es- 
sentially a product syllabus with skills focus, would therefore maintain a fo- 
cus on the main skills that were regarded as most needed to be improved by 
the students and the lecturers from a proficiency perspective, in other words 
writing and speaking, and there would also be a concurrent focus on reading 
and to a lesser extent on listening, given their frequency of use and therefore 
relative importance. Significantly, however, the redesigned syllabus would 
focus on providing students with more opportunities for engaging in the pro- 
ductive skills; in the case of speaking, there would be more opportunities to 
engage in spoken interaction, especially student-initiated interaction as well 
as more prolonged student-student interaction. This aim evolved not only 
from the reported frequency of use of speaking in the classes and the relation- 
ship that was demonstrated between the frequency of practice and self-re- 
ported proficiency levels, but also from the findings’ clear indication that stu- 
dents were probably not provided with opportunities for much, if any ex- 
tended speaking time in their EMI classes. As far as writing was concerned, 
the new syllabus would provide for more individually focused writing activ- 
ities to provide more relevant writing practice. Moreover, the redesigned syl- 
labus, given my beliefs as a teacher and my desire to adopt an approach that 
would better foster learner autonomy and be more learner-centred, would 
also incorporate more opportunities for students to participate in the decision- 
making aspects of the ESAP course, and would therefore introduce elements 
that could be negotiated. This process approach to syllabus design, which in- 
troduces negotiation into the syllabus, would therefore act as a means to pro- 
vide opportunities “for authentic language use about matters that are of im- 
mediate significance to learners” (Breen & Littlejohn, 2000c, p. 19) and so 
would provide opportunities for extended speaking time and therefore fur- 


ther opportunities for language and skills improvement as well as allowing 
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students to participate in some of the decision-making aspects of the course. 
Consequently, the process approach that used negotiation would directly ben- 
efit the skills-based approach to the syllabus as the negotiation would provide 
concrete opportunities for the students to engage in focused and relevant 
skills practice (Prior, 2021). 

Given the fact that a syllabus that has a skill-based approach is regarded 
as a product approach to syllabus design, but the negotiated syllabus is con- 
sidered a process approach to syllabus design, the redesigned syllabus there- 
fore required a blended approach that would be able to integrate aspects of 
both the product and process approaches to syllabus design. The following 
section will now present a critical evaluation of the redesigned syllabus that 
was implemented in AR Cycle 2 that introduced negotiation to provide stu- 
dents with both a greater say in their course as well as opportunities for more 
skills practice. 


7.3 The Redesigned Syllabus Used in AR Cycle 2 
(Syllabus 1) — Evaluation and Discussion 


This section will firstly provide an overview of the ESAP course and its con- 
text within the Faculty and its undergraduate programmes, and will then pre- 
sent the redesigned syllabus, in light of the fourth research question. The sec- 
tion will proceed by detailing the implementation of the redesigned syllabus, 
and provide a critical evaluation of the implementation, referring also to the 


fifth research question. 


7.3.1 Context of the ESAP course 


The ESAP course that is the subject of this volume was, and still is, a 30-hour 
course incorporated into the Faculty’s two undergraduate degree pro- 
grammes, E&M and PPE, and which I have taught since its inception in 
2003/2004. The ESAP course is a compulsory course of each undergraduate 
programme and is worth 3 credit points (following the European Credit 
Transfer System — ECTS). As the University is officially trilingual and offers 
most of its undergraduate degrees in German, Italian and English in varying 
combinations, all students who do not have English as their official L1 have 
the ESAP course inserted into their study plan. The course is timetabled in the 


199 


Integrating Classroom-Based Negotiation Into a Syllabus 


first semester of the second year of PPE and in the second semester of the sec- 
ond year of E&M, so the two versions of the ESAP course do not take place 
concurrently. The ESAP course comprises one group for PPE students and is 
split into two groups for the E&M students given the larger numbers enrolled 
for the latter degree course. The classes for the two E&M groups occur in dif- 
ferent time slots given that I teach both groups. Attendance is not compulsory 
at the Faculty so there is no requirement for students to be present in class but 
all students who have the ESAP course in their study plan must eventually 
pass the course’s final exam in order to complete their undergraduate pro- 
gramme and graduate. The exam for the course, like all the courses at the Fac- 
ulty, is offered three times a year. First semester courses have exam sessions 
in January/February, June/July and September. For courses that take place in 
the second semester, the three exam sessions offered are June/July, September 
and January/February of the following academic year. The vast majority of 
students sit the exam in the exam session immediately after the course fin- 
ishes, but the other sessions are available for students who are unable to sit 
the exam immediately, often due to being on exchanges, or for those who fail 
the exam in the first session. However, students are allowed to sit the exam a 
maximum of twice in an academic year. 

The syllabus that was being used for the ESAP course, as mentioned 
previously in this volume, was a mainly product syllabus based on skills im- 
provement. The course had been designed to provide skills practice in writing 
and reading particularly, which was reflected in how the exam had been de- 
signed. The exam, like all the other exams at the Faculty, was required to con- 
tain a written component, but given that the ESAP course is officially classi- 
fied as a language course rather than a content course, an oral component was 
also required. These requirements had been established at the inception of the 
undergraduate programmes and acted as constraints for this context. There 
had been no indication from the Faculty as to the weighting of the various 
components of the exam, and so the written exam had been allocated 50% of 
the final mark and the oral exam 25%. The final 25% of the mark was allocated 
to a Portfolio, which is work devised at my discretion that the students com- 
plete during the semester as coursework and then submit before the official 
exam session starts. A deadline is set for the Portfolio submission, which is 
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usually three weeks before the date of the written exam in each exam session, 
in order to provide enough time to correct all the work completed for the Port- 
folio before the students have to sit the oral exam. Table 7.1 shows how the 
marks are allocated to the various components of the ESAP course’s exam: 


Table 7.1 — Components of ESAP course final exam 


Exam component Value Points 
Written exam 50% 15 
Portfolio + oral exam 25%+25% 7.5+7.5 


The second column in Table 7.1 shows how the points were allotted to each 
component. Exams at Italian universities are marked out of a total of 30 points 
with a minimum pass mark of 18. An exceptional performance in an exam can 
be awarded with the cum laude distinction so a mark of 30 con lode (cum laude 
in Italian) is also available. As is common practice in Italian language exams 
held at university level, the written exam is scheduled before the oral exams 
and the students must pass the written exam to be admitted to the oral exam. 
A pass of the written exam is understood to be 60% or more. In my exam, 
students therefore needed to pass the written exam with at least 60% and then 
do the oral exam. If the oral and the Portfolio, which were equally weighted, 
then amounted to at least 60% together, the student would pass the exam. 

The written exam was divided into two parts, where Part A was 
comprised of various language exercises and Part B was a writing task. The 
Portfolio that was being used until the redesigned syllabus was introduced 
was based on the book Freakonomics by Levitt & Dubner and comprised 
questions on the contents of the book, language work and a summary of one 
of the topics available to be chosen for the oral exam. The Oral exam 
comprised a 5-minute presentation on the topic chosen from the list contained 
in the Portfolio. 


7.3.2 The redesigned syllabus — Negotiated elements 
(Research Question 4) 


The redesigned syllabus was used for the first time with the E&M students in 
the period from February to July 2015, which corresponds with this study’s 
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AR Cycle 2. Due to there being no compulsory attendance at the Faculty, there 
is no record of how many students attended the course during that timeframe, 
but a total of 107 students sat the exams over the three exam sessions. There- 
fore, it is likely that there were usually at least 35 students present in each 
group during the course as that was the average number that had been attend- 
ing around that time. 

As discussed in the previous section, due to the findings that had origi- 
nated and were merged from the analysed data collected from the students 
and lecturers in AR Cycle 1, the redesign of the syllabus envisaged maintain- 
ing the skills focus of the previous syllabus with the added intention to in- 
crease the opportunities for practising the productive skills through the intro- 
duction of negotiation. However, it was necessary to decide which elements 
could be negotiated with the students, considering the constraints of the con- 
text and experiences in other contexts, thus providing an answer to the fourth 
research question. 

As was discussed in Chapter 2, the literature about process or negotiated 
syllabuses often tends to stress the negative aspects of negotiation, such as its 
radical nature (Bloor & Bloor, 1988; Clarke, 1991), resistance from the students 
themselves (Bloor & Bloor, 1988; Budd & Wright, 1990; Newstetter, 2000; 
Slembrouck, 2000; Smith, 2000; Sokolik, 2000) and resistance from the institu- 
tion (Slembrouck, 2000; Breen, 2001). Furthermore, many of the admittedly 
few accounts of negotiation being used in the classroom in university and 
other tertiary contexts report total failure, or they had not achieved particu- 
larly positive results (e.g. Slembrouck, 2000; Ivanič, 2000). As a consequence, 
although I was committed to introducing negotiation due to its many positive 
aspects such as it being a more democratic way to work and its potential to 
make the learning process more relevant and meaningful for each individual 
student, I felt that whatever type of negotiation I introduced should be on an 
optional basis for the students in an attempt to address some of the potential 
pitfalls of negotiated syllabuses that have been reported. If it were to be op- 
tional, therefore, this would imply that negotiation of the content and proce- 
dures of classroom work would be precluded, as any negotiation would inev- 
itably have to be applied to all attendees of the class or course. Moreover, as 
there was no scope to change the assessment methods, insofar as the exam had 
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to comprise a written and oral component and these had to be the same for all 
students throughout the academic year in order to comply with the Faculty’s 
examination regulations, the main change to the syllabus would have to in- 
volve the Portfolio, the assessment component that I, personally, had intro- 
duced to the course. This was the only part of the course that was flexible 
enough to be able to be negotiated with individual students on an optional 
basis and so the aim was to provide an opportunity to negotiate the contents 
and procedures of the Portfolio for any student who chose to do so. If students 
chose not to take the option of negotiating the Portfolio, they would be able to 
complete the Freakonomics Portfolio, as it became called, which worked to the 
same model as the one of the previous academic year and was provided by 
me in parallel with the Negotiated Portfolio, as the new version came to be 
called. 

Consequently, as far as the fourth research question is concerned, “What 
elements of the syllabus can be negotiated with the learners considering the 
constraints of this particular context and experiences in other contexts?”, the 
element of the syllabus that was able to be negotiated was all the work con- 
tributing to the Portfolio, the coursework element of the final exam. This ne- 
gotiation would be done on a one-to-one basis with individual students who 
chose to do the Negotiated Portfolio due to my reluctance to impose the con- 
cept on all of the class members given the many negative reports and out- 
comes of negotiated syllabuses in the literature. 


7.3.3 The redesigned syllabus — Implementation and discussion 


The procedure to introduce the Negotiated Portfolio was relatively straight- 
forward and followed the basic process as introduced by Breen & Littlejohn 
(2000c) in their negotiation cycle which was discussed in Section 2.4.2 and vis- 
ualised in Figure 2.1. The concept of the Negotiated Portfolio was introduced 
to the students in the introductory class when the course was presented and 
they were told that the negotiated decisions about the Portfolio concerned the 
contents, ways of working and evaluation. A document outlining the guide- 
lines was then made available on the course’s section in the University’s learn- 
ing platform with the procedure that would need to be followed for students 
who intended to take up the option of the Negotiated Portfolio. This document 
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was produced in order to facilitate the negotiated element of the syllabus and 
“to provide a framework for decision-making during teaching and learning in 
a classroom setting” (Breen & Littlejohn, 2000c, p. 29). In practical terms due 
to the skills focus of the syllabus, students were able to select any reading text 
as a basis for the work for their Portfolio. They then had to write me an email 
following the guidelines that were provided for them. Once they had written 
to me, I contacted them and we arranged a meeting to have a follow-up dis- 
cussion to negotiate the contents, was of working and evaluation means. The 
intention behind this negotiation was to provide students with initial skills 
practice in writing the proposal and negotiating the Portfolio with me face-to- 
face. The focus on selecting a text would ensure opportunities for extensive 
reading, an activity which has been shown to “[lead] to significant improve- 
ments in many language skills: listening, grammar, spelling and writing” 
(Grabe, 2009, p. 324) and which is an efficient use of time in a context where 
class time is limited since “no other set of reading activities or reading practice 
can substitute for reading a longer text with reasonable comfort and without 
needing to stop constantly, and without feeling fatigued or overwhelmed” 
(Grabe, 2009, p. 311). Moreover, given the amount of reading the students had 
identified they needed to do at the Faculty for their EMI courses, and the fact 
that “people learn to read, and to read better, by reading” (Eskey, 2005, p. 574) 
extensive reading was deemed the most beneficial and practical approach to 
reading in this context. Consequently, allowing the students to choose a text 
which would be of interest and relevance to them, and thus providing them 
with “course-specific motivational components” (D6rnyei, 1994, p. 277), 
would lead to greater motivation to read the chosen text and thus acquire the 
language benefits from the reading. Once the text had been read, further spe- 
cific writing practice would be provided during the completion of the Portfo- 
lio depending on what type of writing tasks had been negotiated between the 
student and me. Further speaking practice would be provided in the prepara- 
tion for the oral exam, which would be based on the contents of the Negotiated 
Portfolio. The form of the oral exam (a presentation or more informal discus- 
sion, for example) was also open to negotiation in the meeting. 

The intention for this version of the Negotiated Portfolio was therefore to 
provide students not only with a voice in the decision-making processes of 
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their course where they had the opportunity to choose an element of the 
course that would be relevant for them, within the constraints of the context 
and my reservations as noted above, but also to provide extra skills practice 
in extensive reading as well as in writing and speaking with feedback from 
me as the teacher. Consequently, this version of the Negotiated Portfolio com- 
plied with Clarke’s definition of a negotiated syllabus which “allows full 
learner participation in selection of content, mode of working, route of work- 
ing, assessment and so on” (1991, p. 13) and was able to be accommodated 
within the constraints of the context. 

This version of the negotiated syllabus was implemented and was com- 
municated to all the students in the first class and the support material was 
made available on the University’s online learning platform. However, two 
problems became immediately apparent. The first was that this version of the 
Negotiated Portfolio, although adhering to many of the precepts that underlie 
a negotiated syllabus, did not adhere to perhaps the most important of the 
precepts that is clearly referenced in Breen & Littlejohn’s main definition of 
negotiation, which is “discussion between all members of the classroom to decide 
how learning and teaching are to be organised” (2000b, p. 1, emphasis 
added). The Negotiated Portfolio in this redesigned syllabus totally disre- 
garded the classroom focus of negotiation since the only envisaged negotia- 
tion was between the student and me, the teacher, and did not foresee any 
useful student-initiated interaction or student-student interaction that had 
been identified as largely lacking in the EMI classes and whose introduction 
into the classroom was an aim of this study. Secondly and more prosaically 
perhaps, but certainly extremely impactful for this study, the Negotiated Port- 
folio was almost entirely ignored by the students, and in fact only two actually 
decided to take the opportunity to negotiate their Portfolio with me. 

The two students that did opt to undertake the Negotiated Portfolio du- 
tifully contacted me with a written proposal outlining the texts on which they 
wanted to focus and how they wanted to exploit those texts. I then scheduled 
a meeting with them and we agreed on the contents and procedure for their 
individual Portfolios. They worked on their Portfolios and then submitted 
them for evaluation following the general procedure for Portfolio submission 
that had been devised for all the students who had chosen to complete the 
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Freakonomics Portfolio and the presentation that they used for the oral exam 


was assessed in the same way as the other students’ oral exams. 


7.3.4 The redesigned syllabus — Evaluation and discussion 


Due to the AR cycles that characterised this study, evaluation of the action 
that had been enacted was conducted at the end of the course in AR Cycle 2. 
Given the vast majority of students had not chosen to complete the Negotiated 
Portfolio, it was imperative to understand why there had been such a lack of 
interest in the alternative to the Freakonomics Portfolio and to devise a new 
plan to overcome this problem. 

In order to obtain feedback from all the students who had chosen not to 
negotiate the contents of their Portfolio, therefore, a paper-based question- 
naire was distributed at the beginning of their written exam in order to gain 
as many responses as possible. The procedure for this analysis and the results 
were discussed at length in Section 5.6.1.1, and the main findings were that 
the students generally felt that reading Freakonomics, the basis of the alterna- 
tive to the Negotiated Portfolio, was more interesting and less time-consum- 
ing than negotiating their own content. Consequently, it seemed clear that if 
more students were to participate in the Negotiated Portfolio, and thus benefit 
from the opportunities to participate in the decision-making processes of the 
course, how the source material was to be chosen would have to be reassessed. 

Although learner autonomy implies students have a choice in their learn- 
ing (Cotterall, 2000), which can clearly extend to choosing the materials and 
tasks to be done (Breen & Littlejohn, 2000a; Little, 2009), it seemed that the 
students had not been ready for such a degree of choice. Indeed, in the evalu- 
ation of the failure of the negotiated syllabus recorded by Slembrouck (2000), 
Breen & Littlejohn suggest that because the amount of freedom those particu- 
lar students had been given was so much more than what they had been ac- 
customed to, “the ‘leap’ they were required to make was too great” (Breen & 
Littlejohn, 2000d, p. 291). As this seemed to be the case also in my study, I 
decided it would have to be me who made the decision about the source ma- 
terial, whether that was Freakonomics or something else. Although this funda- 
mental decision concerning the contents of the Negotiated Portfolio would 
therefore be taken away from the students, “teacher control” (Illés, 2012, 
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p- 508) is needed in education, even when the context is learner-centred, 
which aims to enhance learner autonomy. Moreover, given the constraints of 
my context, with the short course length and the relatively large number of 
students that were doing the course, from a pragmatic perspective it seemed 
reasonable to take that decision myself. 

A further aspect that arose while reflecting on the implemented syllabus 
was that the negotiation used in AR Cycle 2 had not involved classroom-based 
negotiation, which had been one of the aims of the study in order to provide 
more student-initiated interaction or more prolonged student-student inter- 
action. Therefore, the modified syllabus to be used in AR Cycle 3 would also 


have to include a clear intent to include classroom negotiation. 


7.4 Discussion of Findings From the Data Analysis in 
AR Cycle 2 — Questionnaire (2 & 3) 


Concurrently with the end of the ESAP course that took place in AR Cycle 2, 
in May 2015, the second online questionnaire was conducted with the students 
in order to gather further data from the students so as to compare with the 
data gathered in AR Cycle 1 in order to provide a fuller picture of the language 
learning needs of the students at the Faculty of Economics. A questionnaire 
was also conducted in AR Cycle 3 with the same aim. 

The findings from both questionnaires tended to corroborate many of the 
results from the first questionnaire; all the skills were practised by the stu- 
dents at the Faculty in the second and third year of data gathering, and the 
skills most needed by the students from a frequency perspective were still 
reading and listening. However, as the data from the first questionnaire 
showed, students felt their productive skills were weaker than their receptive 
skills and writing had the lowest mean, but speaking was the weakest consid- 
ering the SD. There was no significant relationship between frequency of prac- 
tice and proficiency level for writing, but there was a significant relationship 
for the other three skills. These findings therefore confirmed that the revised 
syllabus’s focus on providing more opportunities for speaking and writing 
practice was appropriate for the next AR cycle and that a focus on reading 
would still be consistent regarding the amount of reading students stated they 
undertook at the Faculty. 
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7.5 The Modified Syllabus Used in AR Cycle 3 (Syllabus 2) 


The following section will now evaluate how the syllabus was modified in AR 
Cycle 3 in light of the lack of student interest in the Negotiated Portfolio that 
was observed in AR Cycle 2. 


7.5.1 Focus of negotiation 


Given the problems that had been encountered during the implementation of 
the redesigned syllabus in AR Cycle 2 relating to the small number of students 
electing to do the Negotiated Portfolio as well as the unsatisfactory execution 
of the negotiation, I introduced modifications to the Negotiated Portfolio for 
the following course, which took place between October 2015 and January 
2016 with the PPE students. The intention was that the whole class would par- 
ticipate in the negotiation, which would provide opportunities for student- 
initiated interaction and more prolonged student-student interaction as well 
as interaction with me. As the PPE group was smaller than the E&M students, 
as discussed in Section 6.3.1, I felt the negotiation with the whole class would 
be less complicated to organise than with the two larger groups of E&M stu- 
dents that would attend their course from February 2016. Consequently, the 
negotiation with the PPE students would act as a trial run for the second se- 
mester’s course and any difficulties could then be verified and solutions could 
be developed before the large E&M classes starting in February 2016. 


7.5.2 The modified syllabus — Negotiated elements 


For the modified syllabus implemented in AR Cycle 3, therefore, although the 
contents to be negotiated would remain the same, i.e. the work to be done for 
the Portfolio, most of this work would this time be based on the book 
Freakonomics, given the findings from the survey conducted with the students 
at the end of AR Cycle 2 and detailed in Section 7.3.4. 

Moreover, the Portfolios in previous years had also included a require- 
ment to write a summary of the topic that had been chosen for the presentation 
that would be given for the oral component of the final exam. I decided to 
retain this writing task, but the topics would be opened up to negotiation, 
given that “learners have special rights when it comes to deciding the content 
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of courses they are to undergo” (Long, 2005c, p. 26). From a pragmatic per- 
spective, moreover, because the students were already in the second year of 
their studies, they would be familiar with the content of their own study plans 
and were thus “experienced ‘in-service’ informants [who] often make excel- 
lent sources on the content of their job, training course, field of study, 
etc.”(Long, 2005c, p. 27 original italics). Further, as the aim was to have a 
learner-centred focus to the syllabus, the students would obviously have their 
own interests regarding the content of these topics. All these considerations 
indicated that the students were the best sources to provide input concerning 
the topics that would be useful or relevant for the Portfolio element of their 
ESAP course. 


7.5.3 The modified syllabus — Implementation and discussion 


The implementation of the modified syllabus in AR Cycle 3 took place initially 
in a similar way to the syllabus used in AR Cycle 2. The students were pre- 
sented with the concept of the Negotiated Portfolio in the first class where the 
general course contents and assessment means were presented. They were 
also instructed to obtain a copy of Freakonomics and read the contents pages 
and introduction of the book by Class 4 of the course, which would be dedi- 
cated to negotiating the contents and procedures relating to the Negotiated 
Portfolio. These instructions were also made available in the course’s section 
on the University’s learning platform for any non-attending students as re- 
quired by the Faculty. They were reminded about the Negotiated Portfolio 
class in all the other classes leading up to Class 4. 

On the day of the Negotiated Portfolio class, there were initially 12 stu- 
dents present, as can be noted from the field notes that were written up fol- 
lowing the class. The students were provided with the in-class worksheet and 
the procedure followed using the worksheet was detailed at length in Section 
4.7.1.3. The students were initially asked to work individually to complete 
Section A of the worksheet since it was designed to promote self-reflection 
about their strengths and weaknesses in English, and then, according to my 
field notes, they worked in groups of 3 or 4 for 45 minutes discussing the ques- 
tions contained on the worksheet. These questions provided them with the 
framework to discuss the contents of the Negotiated Portfolio based on 
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Freakonomics, i.e. which chapters they wanted to focus on and what kind of 
writing tasks they felt they would benefit from completing, as well as the pro- 
cedures connected to the Negotiated Portfolio, especially with a focus on how 
much work the Portfolio would provide. They had also been given a copy of 
the Portfolio used the previous academic year to provide a further contextual 
framework and example that they could use as a stimulus for discussion, alt- 
hough it was made clear to them that it did not have to be used as a model for 
their own Portfolio. Once they had had an opportunity to discuss the ques- 
tions on the worksheets in their small groups, they came back as one group, 
which had increased to 15 people, and together we went through their an- 
swers to the questions on the worksheet, thus negotiating the contents and 
procedures of the Negotiated Portfolio for their course. 

From the field notes that I wrote up immediately after the class, it is ap- 
parent that the students were given ample opportunities for prolonged stu- 
dent-student interaction during the second phase when they were working in 
small groups as I noted that “there seems to be quite a lot of discussion”. 
Moreover, I wrote that “most have worked up to the time limit of 11.30” thus 
showing that they had spoken for the whole time allotted to them with only 
minimal interaction with me since “I went round once to make sure every- 
thing was clear”. During the group discussion with me, I noted afterwards 
that there “was a good discussion involving most of the students” and that 
“there was prolonged discussion” and some parts “did provoke a certain 
amount of discussion” so there were clearly also many opportunities for the 
students to speak during the negotiation together and with me. In the reflec- 
tive section of the field notes at the end, I noted the following about phase two 
of the worksheet where the students were engaging in student-student inter- 


action: 


I felt that the two-hour session went very well. The students were clearly ready to 
talk about the book as most of them had brought their copy with them and had it 
there on their desks. They started off relatively quietly at the beginning and I was 
afraid that their preparatory discussion in pairs or threes would fizzle out quite 
quickly, but in the end most of the groups kept on talking the whole time allotted 
for that part of the class (from 10.55 to 11.30). 
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This extract highlights the fear I had that the students would be reticent with 
their speaking, and in fact it is clear that there was reticence at the start of the 
student-student interaction. However, the notes reflect that the groups did 
manage to continue with their discussions for the time allotted, which was just 
over half an hour, and it is likely this was largely due to the fact that they were 
prepared for the task: most had a copy of the book and had read the introduc- 
tion, so they already had an idea as to the contents of the whole book and the 
worksheet had provided them with a clear framework that aided the structure 
and contents of the discussion. 

The field notes also referred to the whole class discussion phase with me 
as follows: 


The whole discussion from 11.30 to 12.30 went exceedingly well. All of them seemed 
motivated to contribute and listen and I didn’t sense any boredom or waning atten- 
tion. Some obviously spoke much more than others but in the end everyone con- 
tributed to the discussion. It was good to see that they were all engaged with the 
topic and that we did manage to negotiate to a decision which I hope they are all 
happy with (Prior, 2018). 


What is interesting to note from this extract is the reference to the fact that all 
the students contributed to the discussion, which was a specific aim of the 
task. It is true that there were only 15 students present in this particular class, 
which clearly provided more opportunities for each individual to participate 
than if there had been many more present. However, the references to their 
engagement with both the content of the discussion and the fact that they were 
negotiating aspects related to the work they themselves would have to com- 
plete would suggest that it was these aspects that provided the motivation for 
them to contribute to the discussion. This assumption was in fact supported 
by some of their comments in the evaluation of the new syllabus that was ef- 
fected at the end of the course and which will be discussed in the following 
section. 

The extract from the field notes above also makes reference to the negoti- 
ated elements of the Portfolio and the outcome that was achieved. The stu- 
dents had decided they preferred to focus on chapters 1, 2 and 3 of the book 
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(out of a total of six chapters) given their contents and therefore not to read 
the entire book. This differed from the previous year’s Portfolio, which had 
required the students to read all the book and answer questions on each chap- 
ter. In the field notes, I make a reference to this reduction in the workload, 
stating: 


They managed to bring down the Portfolio’s workload by a good third, which I 
hope will also contribute to less criticism in the end-of-course evaluation, which has 
been characterised in previous years with criticism that the course requires them to 


do too much work for the number of credit points earned. 


The issue of workload for the ESAP course had arisen in previous years as 
many students felt there was too much work required for the 3 credit points 
allotted to the course. Indeed, in the evaluation undertaken at the end of AR 
Cycle 2 to investigate why almost all the students had not opted for the Ne- 
gotiated Portfolio, the second most popular response was number 4, “I 
thought it would be too much work” (see Section 5.6.1.1 and Table 5.10). It has 
been suggested that students’ perception of workload can be influenced by 
curriculum choices, and “an approach to teaching which requires active en- 
gagement of students” (Kember, 2004, p. 181) can lead to students being more 
satisfied with their workload. Therefore, allowing the students to make deci- 
sions about the amount of work to be undertaken for the Portfolio was a 
means not only to reduce the criticism directed towards my course, but also 
to promote greater motivation in the class in general. Reference to this aspect 
was again noted in the students’ end-of-course evaluations. 

Regarding the questions on the worksheet about the language tasks for 
the Portfolio, and therefore the other skills that the Portfolio would foster, the 
field notes make reference to the students’ uncertainty about this part of the 
task. Although this question “generated a certain amount of debate”, this was 
due to “the difficulty for the students to articulate what types [of writing 
tasks] they would find useful’. Long refers to using learners as sources for 
making decisions about courses and although he claims they are useful to pro- 
vide information on the content, they are “inadequate when it comes to intui- 


tions about their language needs” (Long, 2005c, p. 27). 
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To summarise, the field notes I wrote up immediately after the Negotiated 
Portfolio class that took place with the PPE students indicated that they had 
been provided with many more and longer opportunities to practise their 
speaking. Further, the field notes noted that there was a positive atmosphere 
in the classroom as the students had engaged fully with the negotiation. The 
outcomes also seemed positive: the overall workload had been reduced quite 
significantly, and the students had chosen not only the aspects they wished to 
focus on in Freakonomics but they had also provided input for the topics that 
could be chosen for the oral exam. 

A further aspect that emerged from the analysis of the in-class worksheets 
that were used with the PPE students, and again with the E&M students in 
the following semester’s class, was the responses to Section A of the work- 
sheet, the analysis of which was described in Section 5.6.2.1. Section A re- 
quired students to reflect on their strengths and weaknesses in English and 
the top strengths listed were the receptive skills and grammar, and the main 
weaknesses were speaking, vocabulary, especially “advanced”, “specific” or 
“formal” vocabulary, as the students had qualified the term, and grammar 
(see Table 5.13). These responses generally corresponded with the data that 
had been collected from the questionnaires regarding the strengths, and alt- 
hough speaking was listed as the main weakness, which also corroborated the 
findings from the questionnaires, the focus on vocabulary was an aspect that 
had not been rendered explicit in the questionnaires due to their focus on the 
language skills. However, not only were these responses useful as a further 
means to verify the findings from the questionnaire data from the students, 
but they also confirmed some of the findings from the data analysed from the 
EMI lecturers’ interviews. As discussed in Section 5.6.2.1, some responses 
from this section of the in-class worksheet confirmed that anxiety was a factor 
in causing problems for students when speaking, and this aspect was men- 
tioned explicitly by two students: “I feel anxious when I have to speak in front 
of a class” and “anxiety reduces my profitability [sic] in speaking”. 

The Negotiated Portfolio was therefore prepared and put onto the learn- 
ing platform for all the students to access throughout the duration of the ESAP 
course. The course then provided further exercise classes where the aspects of 
Freakonomics chosen to be included in the Portfolio were discussed, thus 
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providing more focussed student-initiated interaction. These exercise classes 
were organised so that the groups were smaller than the main lectures in order 
to provide as many opportunities as possible for students to engage in speak- 
ing practice, given the syllabus’s focus of providing more opportunities for 
student-student interaction. The classes focused on the main themes of the 
chapters that had been chosen to be read for the Portfolio, and various fluency- 
focused speaking activities were incorporated in order to discuss and explore 
those themes. These classes also provided practice in some of the specific writ- 
ing skills that were needed for the completion of the Portfolio. There was a 
particular focus on practising writing short paragraphs that were required for 
the completion of question 1 and 2 of the Portfolio whereas in the lectures 
students were provided with activities and practice in preparation for the 
longer writing text required for the Portfolio in question 3, as well as for the 
academic report required for the written exam. Although the specific writing 
tasks contained in the Portfolio had not been open for negotiation and were 
designed by me, the content focus was always what had been agreed upon 
together in the Negotiated Portfolio class. 

A further consideration concerning this Negotiated Portfolio is that 
although only 15 students had been present during the Negotiated Portfolio 
class, the Portfolio that was produced had to be completed by all students, 
regardless of whether they had been present that day or not. In total, 25 PPE 
students sat the exam in that academic year, so at least ten had not been 
present when the Portfolio was negotiated. Although this may seem to 
contradict the essence of the learner-centred aspect of being able to contribute 
to some of the decision-making elements of the course, I had to make this 
choice for pragmatic reasons. It is a requirement of the Faculty that non- 
attending students of any course have to sit and pass the same exam that 
attending students are required to do, which implied that the Portfolio 
component had to be the same for all students who wished to sit the ESAP 


course’s exam. 
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7.5.4 The modified syllabus — Initial evaluation 
(Research Question 5) and discussion 


The evaluation process for the modified syllabus that was used in AR Cycle 3 
for the PPE students comprised an end-of-course survey that was adminis- 
tered at the end of the course in February 2016. This survey was primarily 
designed in order to provide information for the study’s fifth research ques- 
tion, which was: “What evaluation methods can be devised to ascertain the 
effectiveness of this proposed syllabus for each individual learner?” 

The survey requested feedback on whether the students felt using a Ne- 
gotiated Syllabus was beneficial and their responses to that question and the 
analysis undertaken were detailed in Section 5.6.1.2. As only 15 responses 
were collected from the PPE students given it was such as small group, they 
were amalgamated with the end-of-course surveys that were conducted with 
the E&M students in July 2016 and were analysed together at the end of AR 
Cycle 3 in order to obtain a broader perspective on the syllabus implementa- 
tion. The findings from this evaluation and the implications concerning the 
fifth research question will be discussed in Section 7.5.6. 

However, the end-of-course survey was also conducted with the PPE stu- 
dents for pragmatic reasons given the fact that another problem had arisen 
during the course; erratic attendance. As the University does not require com- 
pulsory attendance of any course, students were not obliged to attend the clas- 
ses, but I had noticed a significant degree of irregular attendance during the 
course. As many students had not been attending regularly, this inevitably 
had an impact on the effectiveness of the approach used in the syllabus as it 
affected the amount of classroom language practice with which the students 
were engaging. In addition to the one class where the Portfolio was negoti- 
ated, thus providing the students with many opportunities to engage in stu- 
dent-student interaction, many of the other classes comprised extensive lan- 
guage practice, particularly writing practice given the focus of the course. 
However, if students had not been present in class, they had not completed 
the classroom-based activities and thus had not gained any benefit from the 
practice that these activities provided. The end-of-course survey therefore re- 
quired students to state to what degree they had attended the course and, if 
they had attended less than half of the course, to choose an explanation from 


215 


Integrating Classroom-Based Negotiation Into a Syllabus 


the eight provided. Of the 15 people surveyed, seven admitted that they had 
attended less than half of the course, four had attended between 51-99% of 
the course and only two stated they had come to all the classes (one did not 
provide any information). Of the seven who had attended less than 50%, three 
cited the fact that they had been away from the University for Erasmus or an 
internship, two stated they had work commitments and two had clashes with 
other courses. Consequently, only two students had actually completed the 
course in its entirety, which surprised me greatly as I had assumed that the 
negotiated elements in the syllabus would have provided greater motivation 
to come to class for those who could attend. 

This problem of erratic attendance was also experienced by Slembrouck 
(2000) in his study, as although attendance checks were used in his context, he 
had suspended them since he believed the use of a negotiated syllabus would 
encourage “students to attend sessions because they feel motivated and in- 
volved and not because university regulations stipulate that they must attend 
classes” (2000, p. 145). The inconsistent attendance experienced in his context 
was in fact one of the contributing factors to the failure of his course. As the 
implementation of the syllabus in AR Cycle 2 had not been successful, and the 
modifications to the syllabus in AR Cycle 3 had rectified the problems experi- 
enced, I certainly wanted to avoid any further obstacles that could also make 
my study result in failure as had occurred in Slembrouck’s case. Conse- 
quently, it became imperative to devise a way to improve attendance so that 
the students would be able to benefit from the language and skills practice 
activities that had been incorporated into the course through the modified syl- 
labus. 

Further, as discussed, one of the aims of using the negotiated syllabus 
was to foster greater learner autonomy. Many of the aspects that typify 
courses that have this aim, such as specifying objectives and identifying re- 
sources, as presented by Cotterall (2000), had been addressed through the im- 
plementation of the syllabus following the basic premise of Breen & Lit- 
tlejohn’s negotiation cycle (2000c). However, one aspect that had not been ad- 
dressed so fully was the aspect of “measuring progress” (Cotterall, 2000, 
p. 111), which was also fundamental for the study. Although the end-of- 


course survey had been devised as a means of evaluation in order to address 
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the fifth research question, it was designed from the perspective of Nunan’s 
definition of programme evaluation, which “assist[s] us in deciding whether 
a course needs to be modified or altered in any way so that objectives may be 
achieved more effectively” (1988b, p. 118). From this perspective, the first end- 
of-course survey had investigated why the students had not opted for the 
Negotiated Portfolio in AR Cycle 2 and had provided a very clear answer. 
However, there was no provision in place to ascertain the effectiveness of the 
proposed syllabus from a language proficiency perspective. Nunan refers to 
this perspective as a “product-oriented evaluation” (1992b, p. 185), where the 
outcomes of the learning process are evaluated rather than the process itself. 
Consequently, an approach was required that could both provide information 
about the students’ progress, so that they and I could monitor their 
performance throughout the course, and could also provide incentives for 
attending class more regularly. 


7.5.5 The modified syllabus — 
The use of ECEs: Evaluation and discussion 


The second phase of the implementation of the modified syllabus occurred 
with the E&M students in February 2016. The syllabus that had been used in 
the previous semester with the PPE students and the procedures used in its 
implementation would remain the same. The most significant difference was 
that the E&M course would incorporate extra credit exercises (ECE) to address 
the problem of the erratic attendance in the previous semester and to provide 
further data on the students’ progress as a means to provide some kind of 
measure of the effectiveness of the blended approach (see Prior, 2018). The 
ECEs introduced were based on pop quizzes, whose main aim is to ensure that 
students are continually prepared for class. Pop quizzes are tests that are gen- 
erally administered randomly and without warning throughout a course and 
the uncertainty as to when the tests will take place aims to encourage a more 
regular study rhythm; often when courses have regularly scheduled tests or 
exams, students tend to revise just for the test a few days before (Thorne, 2000, 
p. 204; Graham, 1999, p. 271), which is often ineffective, especially for lan- 
guage learning. Further, pop quizzes tend to penalise students who are not 
prepared, which “undoubtedly contribute to students’ distaste for them” 
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(Thorne, 2000, p. 204). However, it has been demonstrated that doing these 
quizzes improves scores on final exams (Landrum, 2007; Padilla-Walker, 
2006). As my main aim was to encourage students to attend my classes so that 
they would benefit from the additional skills practice introduced, I decided 
that I would avoid the punitive element that typically characterises pop quiz- 
zes and so renamed them “extra credit exercises”. Although still unannounced 
and administered at random intervals throughout the course, the students 
were not penalised if they were not present in class when an ECE was admin- 
istered since any points given were a bonus. Ten ECEs were administered dur- 
ing the course and if students were present in class, they would receive one 
point. If they scored more than 60% on the ECE, they would score another 
point, for a total of twenty points. These points would then be converted into 
a maximum of 2 marks (out of the final mark of 30 awarded for the final exam). 
Therefore, these extra marks rewarded both attendance and positive perfor- 
mance in the ECEs. If a student did not attend any classes and therefore had 
no extra marks, this would not affect their final exam mark as they would 
simply not have any marks to add on as “extra credit”. 

The ECEs were designed initially to test language that had already been 
covered in class, and so comprised matching exercises, short answers to ques- 
tions and vocabulary tests and they lasted between 10 and 15 minutes. As the 
course progressed, however, the ECEs became more complex and involved 
longer written answers and practice in the specific written texts required by 
the course, and these activities could take up to an hour to complete. This in- 
crease in complexity and length of the ECEs provided valuable insight into 
the students’ progress through the course and allowed me to provide focused 
feedback as well as intervene more effectively when problems arose. The stu- 
dents were able to benefit from this feedback as they were able to see exactly 
where they were experiencing problems and take action to rectify aspects be- 
fore the final exam. This can be demonstrated by some of their comments from 
the end-of-course survey that was distributed at the end of the course in the 
evaluation stage of AR Cycle 3, where they were asked to comment on the 
introduction of the ECEs. One stated, “students can see if they have under- 
stood the topics covered during the last lesson” and “it is a good way to re- 


view the contents discussed in class”. 
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Further and most importantly for this study, the ECEs did seem to have a 
positive impact on class attendance. Over the course of the semester there was 
a mean attendance of 57.3%, compared to the previous year’s 48.5%. Even 
though it is impossible to state definitively whether the introduction of the 
ECEs was the direct cause, when the students responded to the end-of-course 
survey on the Negotiated Portfolio, which also included a question on the in- 
troduction of the ECEs, an overwhelming majority of students, almost 90%, 
stated that the use of ECEs was “a good idea”. Many appreciated the fact that 
the ECEs were a way to encourage attendance and comments included, “it 
encourages the students to come to the lessons and to repeat the content at 
home” and “it is a good idea because it incentivises participation and attention 
in class”. Another focussed on the fact that the use of ECEs helped to enhance 
the distribution of study: “it is a good idea because it pushes students to study 
grammar from the beginning and not only before the exam” whereas another 
saw them as a means to provide extra practice opportunities: “you can exer- 
cise during classes and improve”. Some, as I did, also appreciated the fact that 
they provided useful feedback on their progress: “I think this represents a 
good feedback for our work during the course”. 

Although the use of the extra credit exercises seemed to boost attendance, 
unlike others who have used similar methods in the past (Graham, 1999; 
Landrum, 2007; Wickline & Spektor, 2011), the students’ grades did not im- 
prove overall. The E&M course that used the ECEs had a mean grade of 22.6 
(out of a total of 30) whereas the same course the year before when ECEs were 
not used had a mean of 23.2. However, what was significant was that consid- 
erably more students passed the exam after attending the course with the ex- 
tra credit exercises (76% as opposed to 67% the previous year), which may be 
an indication that the ECEs can be seen as being particularly beneficial to those 
students who would have otherwise struggled to pass. This outcome was also 
noted by Graham (1999, p. 272), who found that in his study, it was the mid- 
range students who tended to benefit from ECEs rather than the top students. 

Consequently, the E&M course that took place in AR Cycle 3 and that 
used the modified syllabus used with the PPE students in the first semester 
tended to have fewer problems regarding erratic attendance and more stu- 


dents passed the final exam than the previous year. Although it is not possible 
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to claim these outcomes were a direct result of the introduction of ECEs into 
the course, the students expressed an overwhelmingly positive attitude to- 
wards the use of ECEs and their use also allowed me to monitor the students’ 
progress more closely. 


7.5.6 The modified syllabus — Final evaluation 
(Research Question 5) and discussion 


In order to evaluate the measures put into place using the modified syllabus 
in AR Cycle 3, as discussed previously, an end-of-course survey was con- 
ducted with the PPE students in February 2016 and again with the E&M stu- 
dents in July 2016. The end-of-course survey for the E&M students was iden- 
tical to that for the PPE students but with the addition of question 3 which 
asked for feedback on the use of the ECEs. The responses from the two end- 
of-course surveys were amalgamated and analysed, as detailed in Section 
5.6.1.2, and a conceptual framework evolved based on the responses to ques- 
tion 2 of the surveys. This conceptual framework has been reproduced below 
in Figure 7.1 for ease of reference. 
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The analysis of the responses to question 2 in the end-of-course surveys 
showed that there were two reasons why students felt negotiation was a 
“good” idea. The first was it provided them with an opportunity to express 
their opinion (the category on the left in the conceptual framework) and the 
second was that it made them feel part of the learning process. The ability to 
express their opinion concerned various aspects of their course, from the 
amount of work to be done to the content. As the students in the previous 
years’ courses had complained about the amount of work required for the 
ESAP course, and through the negotiation they had managed to reduce the 
workload of the Portfolio by more than a third, the occurrence of this category 
is unsurprising. Although there is not an extensive literature regarding 
designing a syllabus to reduce workload, it has been suggested that in order 
to produce a syllabus that students perceive has an acceptable workload, 
various approaches can be adopted including “an approach to teaching which 
requires active engagement of students [and] teachers accepting responsibility 
for motivating students and stimulating interest” (Kember, 2004, p. 182). The 
findings from the conceptual framework reflect these two approaches clearly 
since the visualisation in Figure 7.1 demonstrates that the ability to influence 
the amount of work resulted from being engaged in the learning process. 
Moreover, it can be seen that students’ increased motivation was a 
consequence of being able to express their opinions about the workload and 
the course contents, which in turn allowed them to choose content that was of 
interest to them. 

The fact of being able to participate in the decision-making process con- 
cerning content was the main subcategory in this section of the conceptual 
framework, hence its larger size than the “amount of work” subcategory. As 
presented in Section 5.6.1.2, being able to choose content was mentioned by a 
significant number of students in the survey, which corresponds with New- 
stetter’s (2000, p. 184) account of using negotiation in the classroom where 
the favourable reactions from her students tended to focus on the ability to 
exercise choice over the texts they had to write. Furthermore, the conceptual 
framework shows that the students appreciated choosing the content particu- 
larly because it could reflect their interests, which was also clearly evidenced 
in Serrano-Sampedro’s (2000) account of using negotiation in her context. She 
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remarks that due to her learners’ being able to influence the choice of content 
based on their interests, “their sense of progress and achievement increases 
and so does their motivation” (Serrano-Sampedro, 2000, p. 125), thus linking 
exercising choice with higher motivation. In fact one of the strategies sug- 
gested to increase motivation is to “increase students’ interest and involve- 
ment in the tasks by ... adapting tasks to the students’ interests” (Dörnyei, 
1994, p. 281). The conceptual framework in Figure 7.1 therefore reflects this 
relationship between interests and motivation with the secondary connector 
and represents comments from students such as “If students find [the course] 
more interesting, they will have more motivation to do it”. 

The students also expressed appreciation about choosing the content as it 
could be tailored to their needs, which was another frequent reason cited by 
the students but was less frequent than interests, thus its smaller size in the 
visualisation. This aspect has also been reported in another study conducted 
in a school of nursing with qualified nursing staff where using a negotiated 
syllabus “enhanced their motivation by its focus on their professional needs” 
(Martyn, 2000, p. 161). As there were no comments from the students directly 
linking needs with motivation issues, there is no connector between the two 
categories in the conceptual framework, although this does not imply that 
there was not a link between meeting needs and motivational issues in this 
study. 

The second main reason why students felt negotiation was a “good” idea 
was that it made them feel part of the learning process. This aspect, which was 
also coded with the terms involved and participate, has been reported in another 
account of using negotiation in the classroom. In the evaluation of her study, 
Linder (2000, p. 102) states that one of the positive outcomes was that “pupils 
have reported their appreciation of their greater involvement in and with the 
programme”, which led to increased participation in the learning process. 
This feeling of involvement was also observed in a case study where a process 
syllabus was implemented which “revealed that the students were 100 per 
cent in favour of the opportunity to get involved in their course management” 
(Simmons & Wheeler, 1995, p. 63). Although Linder’s study involved second- 
ary school pupils, Simmons & Wheeler’s case study involved professionals, as 
did Martyn’s study (2000). It has been suggested that the positive outcomes of 
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using negotiation in the latter two studies were mainly due to the maturity 
and the relatively high language proficiency of the participants involved 
(Martyn, 2000; Benson, 2001), as well as the fact that the negotiation focussed 
on their tangible professional needs. Similarly, in my study, because they were 
university students, it is most likely that they were more mature than school 
pupils and their overall language proficiency was relatively high. However, 
although they did not have professional needs per se since they were second- 
year undergraduate students, they did have clear academic needs. ESP “is an 
approach to language learning, which is based on learner need” (Hutchinson 
& Waters, 1987, p. 19), but “needs” should not be understood as purely the 
objective necessities that are determined by a target situation; they can also be 


su 


what Hutchinson & Waters refer to as learners’ “wants”, i.e. what they feel 
they need or want from a course (1987, pp. 55-56). As the in-class worksheet 
used in the Negotiated Portfolio class had been designed to elicit the students 
“wants” to a certain degree (see Section 4.7.1.2), the findings from the end-of- 
course survey visualised in the conceptual framework indicate that the use of 
the in-class worksheet was generally effective in eliciting these wants. 

The conceptual framework in Figure 7.1 shows that the “Feel part of the 
learning process” subcategory was further divided into the two categories of 
“Joint decisions” and “Democracy”. The idea of joint decisions being taken in 
a classroom using negotiation either amongst the learners themselves or be- 
tween the learners and the teacher is in fact the essence of process syllabuses; 
Breen & Littlejohn (2000b, p. 2) state clearly that “process syllabuses have 
therefore evolved as a means of planning, implementing and evaluating ne- 
gotiation in the classroom, and the decisions to which teachers and students 
may jointly arrive”. Taking joint decisions implies that the responsibility for 
learning is therefore being shared, an aspect that was evaluated positively in 
the study conducted by Smith (2000) who felt that both the students and the 
teachers benefitted from this aspect. Sharing responsibility also contributed to 
an increase in mutual understanding and “the learners’ views are expressed 
and taken seriously” (Smith, 2000, p. 62). 

The fact that the students’ views were considered to have been taken 
seriously can also be observed in the conceptual framework in the other 
subcategory emanating from the “Joint decisions” category which covers 
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aspects relating to “respect” and “recognition”. There were several comments 
from the students that were included in this category including “interests of 
the students are recognised and respected” and “it gives us the impression 
that the professor is interested in our opinions and takes into account our 
interests and skills”. Given the fact that my ontological perspective is that all 
relevant voices involved in a context have the right to be heard, it is gratifying 
for me as a teacher that this aspect was recognised and appreciated by some 
of the students. 

As discussed in this volume, it has been broadly accepted that learner 
autonomy requires learners to take charge of their learning and take respon- 
sibility for their learning process (Little, 1995; Cotterall, 2000; Benson, 2001). 
Moreover syllabus approaches to learner autonomy have focussed on trans- 
ferring responsibility for aspects of the language learning process from the 
teacher to the learner (Cotterall, 2000, pp. 109-110). From the findings that are 
represented in the conceptual framework, therefore, it is clear that the stu- 
dents felt that being able to negotiate aspects of their course had allowed them 
to contribute to the decision-making processes, and thus provided them with 
more control over their learning. Having more control therefore enables them 
to take more responsibility for their learning, which in turn, according to the 
literature, would lead to increased learner autonomy. It was beyond the scope 
of this study to make any provision to measure attainment of autonomy, 
which has been noted is a generally problematic issue since “autonomy is 
clearly a multidimensional construct” (Benson, 2001, p. 51) and involves a va- 
riety of behaviours that show that learners are exercising control over their 
learning. Even if those types of behaviour are present in a learner, there is 
“little evidence to suggest that autonomy consists of any particular combina- 
tion of those behaviours” (Benson, 2001, p. 51) and in fact “the learner who 
displays a high degree of autonomy in one area may be non-autonomous in 
another” (Little, 1991, p. 5). However, the end-of-course survey in answer to 
question 2 did contain comments that alluded to students’ taking greater re- 
sponsibility for their learning, such as “we can give our input and evaluate 
how much time the course is “worth” for us. It reduces the number of people 
who constantly complain” and this other succinct remark: “Nobody can com- 
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plain afterwards about the content 
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The fact of feeling part of the learning process also produced the other 
subcategory in the conceptual framework, which was “democracy” and is ex- 
emplified with the statement “we have a right to give our opinion”. This sen- 
timent was echoed by several students when giving reasons why using nego- 
tiation was a good idea. One stated “a person should always be able to partic- 
ipate in a discussion and negotiate for desired aims” whereas another re- 
marked “because students are able to give their opinion whether they agree 
on [sic] [the course contents] and the result is based on a democratic fairness”. 
Given that using negotiation in the classroom has been shown to be a highly 
democratic way of working (Breen & Littlejohn, 2000c; Linder, 2000; Slem- 
brouck, 2000), and given my beliefs that the learning process should be organ- 
ised using democratic principles where the main stakeholders should have a 
voice in their own learning process, the acknowledgement from some of the 
students that this indeed had occurred during the course is certainly an ex- 
tremely positive outcome of this project. 

In summary, the end-of-course survey that was conducted with the two 
student cohorts after the modified syllabus was introduced provided a com- 
prehensive evaluation of using negotiation in the course (Prior, 2021). The stu- 
dents were overwhelmingly positive about using negotiation, and of the 105 
completed surveys, only three respondents did not believe using negotiation 
was beneficial. Of these three, only one gave a reason, stating, “I think that the 
professor should decide about the content of the course”. This comment 
demonstrates how traditional educational practices with a clear division be- 
tween the learners and the teacher can still be regarded as the preferred ap- 
proach by the students concerned, which reflects perhaps the greatest oppo- 
sition to negotiated syllabuses and which has been reported in other studies 
(Bloor & Bloor, 1988; Budd & Wright, 1990; Newstetter, 2000; Slembrouck, 
2000; Smith, 2000; Sokolik, 2000). However, since the vast majority of the stu- 
dents surveyed in my study were positive about using negotiation in the 
course and the additional use of ECEs with the E&M students had seemed to 
have had a positive impact on both the attendance and the exam performance 
of those students, I believe that the approach used where negotiation was 


blended with a greater focus on skills development was essentially successful. 
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8. Final Discussion and Conclusion 


8.1 Introduction 


This final chapter will now seek to answer the five research questions that 
were posed in the Introduction to this volume. It will also examine the limita- 
tions of the study, reflect on the learning experiences for me as a researcher 
and will provide an update on the currently evolving version of the ESAP syl- 
labus. 


8.2 The Research Questions 


There were five research questions that informed this study and as presented 
in Section 1.3, they evolved in order to reflect both the AR tradition and the 
convergent parallel mixed methods research design that characterised the 
study. The five question were: 

1. What are the English-language skills needed by economics students at 


this trilingual university as perceived by the main “actors”, i.e. students 
and lecturers? 


2. What skills practice should be maintained or enhanced in the syllabus? 


3. How can a predominantly product syllabus that is skills based benefit 
from the integration of a process approach to syllabus design? 


4. What elements of the syllabus can be negotiated with the learners consid- 
ering the constraints of this particular context and experiences in other 
contexts? 


5. What evaluation methods can be devised to ascertain the effectiveness of 
this proposed syllabus for each individual learner? 


The first two research questions were mixed methods questions given that two 
different data sources were used, the students and the EMI lecturers, and two 
different approaches to data collection were used. The choice of using multi- 
ple sources and methods was motivated by my ontological view that the main 
stakeholders in a course should be given the opportunity to contribute ac- 
tively to the design of that course, a view shared by Auerbach (1995) and Long 
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(2000c). This data collection and analysis functioned as the needs assessment 
and analysis that was required to be undertaken in order to redesign the ESAP 
syllabus since the distinguishing feature of ESP “is not the existence of a need 
as such but rather an awareness of the need” (Hutchinson & Waters, 1987, p. 53 
original italics). Through the questionnaires conducted with the students and 
the semi-structured interviews with the EMI lecturers, two comprehensive 
pictures of the target situation were gained and, following the recommenda- 
tions of Long (2000c), these sources and methods were triangulated to help 
validate the data and to investigate the similarities and differences. As dis- 
cussed at length in Sections 6.1.2 and 6.1.3, the findings from the data analysis 
showed that the students and lecturers were in broad agreement that the stu- 
dents’ productive skills were in most need of improvement. These findings 
led to the design of the revised syllabus in AR Cycle 2, which then did not 
work as planned, yet the findings and further modifications informed the fo- 
cus of the modified syllabus used in AR Cycle 3. This was designed in an at- 
tempt to enhance the opportunities for student-student interaction and stu- 
dent-initiated interaction in the classroom as well as increasing the number of 
writing activities to be done during classroom activities and outside of class 
in the preparation of the Portfolio and provide extensive reading opportuni- 
ties with the use of Freakonomics. The increased opportunities for skills prac- 
tice were introduced due to the findings from the quantitative data analysis 
that showed a significant relationship between the amount of practice stu- 
dents declared they undertook in each skill and their self-reported levels of 
difficulty in the skills; the more they stated they practised the skill the fewer 
difficulties they felt they experienced in this skill. A positive relationship was 
also observed between the amount of practice they stated they undertook and 
their proficiency levels in the skills although this relationship was mainly ob- 
served in the productive skills. 

The third research question focused on the desire to use negotiation in the 
classroom due to my beliefs as a teacher that students should be given oppor- 
tunities to engage in the decision-making processes of their course, not just in 
an initial needs analysis phase, but also throughout the course. Moreover, as 
pragmatism was the worldview informing this study, using negotiation 


would not only allow me to consult the students on the course contents, since 
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the limited number of teaching hours available for the course (30 hours) would 
imply “any choice of curricular contents would be arbitrary” (Slembrouck, 
2000, p. 139), but the negotiation itself would also provide opportunities for 
authentic language practice. Therefore, the choice to use negotiation origi- 
nated from a desire to give the students a greater stake in their learning pro- 
cess and to optimise the relatively limited time available for the course. More- 
over, the fact of providing students with opportunities to use negotiation 
would provide them with more control over their learning process and there- 
fore take more responsibility for their learning, which in turn would enhance 
their learner autonomy. As discussed, measuring learner autonomy is a prob- 
lematic concern but some of the findings from the end-of-course surveys sug- 
gested that some students appeared to have achieved greater autonomy. 

The use of negotiation also provided the students with more opportuni- 
ties to engage in meaningful tasks which provided authentic target language 
practice using student-student interaction, which has been observed to be ben- 
eficial in skills acquisition (Ortega, 2007). In this study, however, the use of 
negotiation occurred mainly in the Negotiated Portfolio class. 

The fourth research question concerned the elements of the syllabus that 
were negotiated with regard to the context and other accounts. Given the con- 
straints of the context, the contents of and the procedures for the Portfolio 
were negotiated with the students. As discussed in Chapter 6, the first version 
of the Negotiated Portfolio did not work satisfactorily due to the students’ 
unwillingness to choose the Negotiated Portfolio over the Freakonomics Port- 
folio. The end-of-course survey conducted with the students to investigate this 
lack of interest revealed that they were disinclined to engage with the Nego- 
tiated Portfolio due to the work involved and their interest in the alternative 
Freakonomics. However, Breen & Littlejohn (2000d, p. 291) suggest that using 
negotiation can provide a degree of freedom that can be too much for some 
students as it is so unusual in standard educational practices, which is a rea- 
sonable explanation for the lack of engagement in AR Cycle 2. As a conse- 
quence, the choice of source material was imposed upon the students in the 
successive AR cycle although the decisions as to how much and which parts 
of the book would be studied was negotiated. 
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In action research, reflection is one of the four main elements in a cycle, and 
evaluation comprises step 3 in Breen & Littlejohn’s negotiation cycle (see 
Figure 2.1) since “the most important characteristic of a process syllabus is 
that it pivots upon the evaluation of an agreed action or set of actions” (Breen 
& Littlejohn, 2000c, p. 33). The fifth research question was therefore designed 
to incorporate these elements in the study. The main evaluation tool com- 
prised the end-of-course surveys that were administered to the students on 
paper and were designed to evaluate “the appropriateness or otherwise of the 
actual process” (Breen & Littlejohn, 2000c, p. 33). The conceptual framework 
that was discussed and illustrated in Figure 7.1 was developed to display the 
findings from the analysis of the responses on the surveys and as discussed in 
Section 7.5.6, the responses from the students were overwhelmingly positive 
concerning the use of negotiation. Moreover, the introduction of the ECEs in 
the E&M course in the second semester of AR Cycle 3 seemed to have a posi- 
tive impact on class attendance and on the overall number of students passing 
the course’s final exam. Consequently, although the evaluation was con- 
ducted with all the group and the responses were aggregated to create the 
conceptual framework, each individual student still had been provided with 
the opportunity to contribute to the contents of the Portfolio through the ne- 
gotiation that occurred in Class 4 (if they attended the class that day), and 
were also provided with a choice of questions to answer in the Negotiated 
Portfolio itself. There was also a choice of topics for the summary that had to 
be written in preparation for the presentation to be given in the oral exam so 


each Portfolio completed was always an individual piece of work. 


8.3 The Contribution of the Study 


This study and the previous publications that have summarised aspects of it 
(Prior, 2020; Prior, 2021), have detailed a rigorous classroom-based investiga- 
tion into the use of a negotiated syllabus in a tertiary setting and as such make 
more recent contributions to the work of other researchers who undertook 
classroom-based research on practical implementations of negotiated sylla- 
buses in university settings such as Martyn (2000), Newstetter (2000) and 
Sokolik (2000), among others. Given that most of these studies were completed 
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more than two decades ago, a more recent investigation can certainly be con- 
sidered timely. 

Further, this research contributes to the knowledge on how using a nego- 
tiated syllabus can be associated with providing learners with greater auton- 
omy in their learning, as was reported by Bloor & Bloor (1988) and Cotterall 
(2000), and how this can be connected to motivation issues, especially relating 
to “course-specific motivational components” (Dérnyei, 1994, p. 277), which 
can contribute to more effective language learning. It has also been suggested 
that using negotiation in the classroom provides more opportunities for skills 
practice and therefore language learning (Serrano-Sampedro, 2000). The re- 
search conducted for this volume has demonstrated that engaging in negotia- 
tion did provide increased opportunities for practice in the target language 
and it is likely this was beneficial for improvement in the skills. 

This study contributes to teaching English in ESP classrooms since alt- 
hough the context was very specific, the approach used can be replicated in 
many secondary or tertiary classrooms, particularly in “low-constraint” con- 
texts (Wette, 2011, p. 137) where teachers have greater autonomy in decisions 
related to syllabus design and content. Using negotiation in the classroom 
does not imply having to negotiate all decisions concerning the syllabus, 
which has been found to be impracticable (Markee, 1997; Breen & Littlejohn, 
2000c; Wette, 2011). As negotiation can be pitched at many different levels, as 
illustrated in Breen & Littlejohn’s curriculum pyramid (2000c, p. 35), it would 
therefore be perfectly feasible for many teachers to initially introduce negoti- 
ation with their learners through a focus on a task or a series of tasks, the first 
levels of the curriculum pyramid. It is highly likely this would provide learn- 
ers with a greater stake in the course, which would mean taking more respon- 
sibility for their own learning and which would consequently bestow a greater 
sense of ownership over their learning process (see Cotterall & Crabbe, 1999). 
Once learners have understood that taking ownership of part of their course 
can be beneficial for their learning, teachers could explore providing them 
with further opportunities to negotiate other aspects of their course, provided 
their contexts provide them with this freedom. 
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Given the fact that there still seems to be opposition from some learners to 
using negotiation in the classroom because it can be regarded as being diver- 
gent from standard educational practices, which implies therefore that learn- 
ers are not accustomed to being involved in the decision-making processes of 
their course(s), if more teachers were to use negotiation in their classrooms, 
even for relatively minor aspects concerning content or procedures, there 
would certainly be more acceptance of it. In my study, most students regarded 
negotiation favourably, which was also observed by Breen & Littlejohn 
(2000d) regarding the accounts of classroom-based negotiation in their vol- 
ume (2000a). Indeed, they state that “process syllabuses and shared classroom 
decision-making represent one of the most significant practical and theoretical 
developments in language teaching in recent years” (2000d, p. 295) although 
since their volume, there have been few accounts of or investigations into ne- 
gotiated syllabuses. The study that is the subject of this volume, therefore, has 
contributed to the literature on practical classroom-based research in ELT as 
it has demonstrated that using negotiation in the classroom was beneficial to 
my students as it provided them with greater ownership of their course, which 
contributed to greater responsibility for their learning. It also provided in- 
creased opportunities for skills practice in the target language, particularly in 
a context where these opportunities were generally lacking. As a consequence, 
there is no reason why this study could not be replicated in other similar con- 
texts. 


8.4 Limitations of the Study 


One of the main limitations of the study is that although the aim was to create 
a more learner-centred syllabus, it was impossible to track individual learners 
in the data collection phase as the questionnaires were anonymous, nor was it 
possible in the end-of-course surveys in the evaluation phases. However, this 
was mitigated in the E&M course in AR Cycle 3 with the institution of the 
ECEs, the results of which were recorded so that the progress of each 
individual student could be tracked. Then I could intervene and provide 
individually focused feedback after each ECE completed. This strategy, 
however, was only effective for the students attending the course. If students 
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chose not to attend the course, or were unable to attend, there was no 
provision in place to compensate for this. 

Another limitation to the study was in fact the seemingly radical nature 
of using negotiation. One of the aims of using negotiation was to provide stu- 
dents with a greater stake in the course and therefore to become more auton- 
omous but as witnessed in AR Cycle 2, only two students took the opportunity 
to negotiate their Portfolio. Moreover, in AR Cycle 3, although most students 
expressed a positive attitude to the use of negotiation, there were a couple of 
dissenting voices, one of which focussed on the fact that they felt I, as the 
teacher should be the one to choose the course contents. This attitude reflected 
a lack of readiness to embrace the greater levels of decision-making that the 
negotiation process afforded them as well as a refusal to accept any method- 
ology that would appear to oppose the prevailing orthodoxy in educational 
practices where the teacher teaches, and the students learn. Successful negoti- 
ation needs the basic premise that there are two willing parties in the negoti- 
ations process. In the case of this study, there were these obvious instances 
when the students were unwilling to participate, which resulted in the nego- 
tiation failing before it had even begun. 

A further limitation is related to the physical and practical context in 
which the ESAP course occurs. In the two Negotiated Portfolio classes with 
the E&M students there were 42 and 34 students present. The lecture halls in 
which the negotiation took place are large halls with fixed tiered rows of desks 
and chairs which makes monitoring problematic. Consequently, although 
many students did speak English in the Negotiated Portfolio class, it was im- 
possible for me to ascertain whether they all used the occasion to speak Eng- 
lish all the time and therefore to benefit from the opportunity to engage in 
prolonged interaction. In the meantime, however, this situation has been ad- 
dressed by my requesting a room with movable furniture for Class 4. This has 
allowed the students to sit in circles and has allowed me to monitor them 
much more effectively even if the number of students has increased since the 


classes in AR Cycle 3. 
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8.5 The Transformative Value of the Study 


Action research has been described as being “transformative” (Borg 2010) and 
even “self-transformative” (Kemmis 2009) and this action research study has 
afforded me with a significant number of opportunities to develop as a re- 
searcher and shape my professional identity as a teacher-researcher. From a 
practical perspective, this study has allowed me to gain numerous research 
skills such as learning how to explore data using various software pro- 
grammes such as NVivo, SPSS and Excel. I have also learnt how to conduct 
interviews and transcribe them and then follow a rigorous coding procedure 
to produce extremely effective conceptual frameworks. I have gained experi- 
ence in writing academic articles through the writing and publishing of arti- 
cles based on this study (Prior 2018; Prior, 2020; Prior, 2021; Prior, 2023) and 
greatly appreciated the learning opportunities provided by the feedback and 
responses that characterised the reviewing process. I have given various 
presentations on aspects of this research and have been gratified that many 
people have shown great interest in the classroom-based research conducted. 

Moreover, by engaging in this research-based study, I have been pro- 
vided with numerous opportunities for developing my professional identity 
as a teacher-researcher. Professional identity concerns “the notion of agency, 
or the active pursuit of professional development and learning in accordance 
with a teacher’s goals” (Beauchamp & Thomas, 2009, p. 177) and it has been 
argued that “teachers undergo shifts in identity through teacher education 
programmes” (Dikilitaş & Yayli, 2018, p. 415). There is broad agreement that 
action research encourages reflective practice (Burns, 1999; Levin & Green- 
wood, 2001; Coghlan & Brannick, 2005; Burgess et al., 2006; Costello, 2011) and 
is even “self-reflective practice” (Carr & Kemmis, 1986, p. 220), which are key 
qualities in professional development. This reflection can then lead to new 
knowledge, new roles and new ways of engaging in practice, which can be 


transformative. Dikilitaş & Yayli state: 


As a truly autonomous professional development activity, action research provides 
potential opportunities for teachers to develop deeper insights, not only into prac- 


tices, but also their existing knowledge through systematic enquiry into the class- 
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room and teaching. In other words, the process of problematizing pedagogical is- 
sues creates a sense of transformation in roles and professional identity. (Dikilitaş 


& Yayli, 2018, p. 415). 


Engaging in this action research study has certainly been transformative for 
me and has allowed me to develop deeper insights into my practice but also 
into my professional identity as a teacher-researcher. I have long believed that 
educational practices should be placed within a democratic framework but 
prior to this study it was often problematic to devise a focused, research-based 
strategy to promote democratic values in the classroom. However, through 
the research undertaken for this study that resulted in the use of negotiation 
in the classroom, I have developed a much more democratic way of working, 
and I can now accommodate the views of the students more easily and more 
frequently into my teaching. 

Moreover, I have developed a greater engagement with the students, 
which was noted as a positive outcome of using action research in Dikilitas & 
Yayli’s study and which they term “developing sensitivity to students” (2018, 
p. 418). Edwards & Burns (2016, p. 11) had also identified this feature of AR 
in their study, claiming it had “led the teachers to establish more open, collab- 
orative approaches to their teaching”. This engagement leads to a greater 
awareness of how students feel about their learning and if problems occur, a 
greater desire to assist them. As prior to this study their complaints about the 
ESAP course’s workload were persistent and frustrating for me, I wanted to 
address the issue but could not envisage how. The introduction of negotiation 
allowed the students to reduce their workload for the Portfolio to a degree but 
more significantly, they no longer tended to complain about it as much as they 
had taken responsibility for the amount of work they had to do as well as the 


contents and procedures. 


8.6 The Current ESAP Course 


At the time of writing, the ESAP course is still running and I am still teaching 
both the PPE students and the E&M students. The syllabus that is currently in 
use is very similar to the one that was used in AR Cycle 3 although certain 
content changes have been made. The Portfolio for the E&M students is still 
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based on Freakonomics but the set book for the PPE students has changed to 
Katrine Margal’s 2021 book Mother of Invention: How Good Ideas Get Ignored in 
an Economy Built for Men in order to provide more recent input. However, both 
Portfolios are still negotiated with the whole class (or the two groups in the 
case of E&M) in Class 4 or 5, and the contents differ each course depending 
on the group’s interests, which influence which topics or chapters are chosen. 
The writing tasks tend to be similar to the syllabus in AR Cycle 3, with a bal- 
anced mix of shorter paragraphs and a longer piece of writing, reflecting the 
texts that were identified as being needed in the EMI courses. The topics used 
for the oral exam, and therefore the summary that needs to be written on that 
topic for the Portfolio, are also still being negotiated with the students and the 
negotiations have contributed to a greater number and variety of topics used, 
which change every year. The ECEs are still being used but the number was 
reduced to seven or eight instead of ten, due to time constraints caused 
particularly by the shift to online learning in the 2019/2020 and 2020/2021 
academic years. The focus of the ECEs has shifted slightly and tends to be on 
writing longer texts. This implies there is now more feedback needed to be 
given, and together with the ever-increasing student numbers, it is likely that 
the ECEs will have to be modified still further in the future. 

Dikilitas and Yayli (2018) found that “engagement in action research was 
also found to influence teacher’s teaching practice” (p. 418), which has cer- 
tainly been the case for me since not only am I continuing to negotiate the 
contents and procedures of the Portfolio with my Economics students, but I 
now do the same with the students of my ESAP course for Design and I started 
this year to do the same for my Computer Science students. I also use ECEs 
with the same Design students, where it has been observed that attendance 


has improved and the exam pass rate has increased noticeably. 
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8.7 Conclusion 


This study has documented a classroom-based application of an intentionally 
blended process-product approach to syllabus design where negotiation was 
used to provide more opportunities for target language skills practice. It has 
demonstrated that this approach can be enacted in a tertiary context success- 
fully and that despite some opposition, it can still be accepted by the vast ma- 
jority of learners. Furthermore, given that in syllabus design, “no one ap- 
proach can be fully responsive to learners’ needs” (Graves, 2008, p. 161), a 
blended approach that aims to draw on and develop some of the strengths 
that characterise process and product approaches to syllabus design can be 
regarded as an effective and efficient approach. Graves states that “the chal- 
lenge is how to identify an organizational structure as a basis for making prin- 
cipled decisions about what and how to teach” (2008, pp. 161-162), and refers 
to the issues with which course developers are confronted, which were iden- 
tified by Snow & Kamhi-Stein (2006): 


This is not a simple or a clean task because it requires synthesizing the massive 
amounts of information gathered through needs assessments, meetings with pro- 
gram administrators and colleagues, review of policy documents and other activi- 
ties. At the same time, in identifying the organizational structure of the course, 
course developers have to take into account logistical constraints, the expectations 
of the educational system in which the course will be offered, explicit and implicit 
teaching policies, the course developers’ own beliefs about teaching and learning, 


and their degree of professional experience. (p. 9) 


This study has shown that using a multicycle action research project to de- 
velop a blended approach to syllabus design, where the course developer is 
the teacher of the course, has been an effective organisational structure to 
manage the numerous challenges that syllabus design and course develop- 


ment inevitably present. 
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